version
##                _                           
## platform       x86_64-apple-darwin15.6.0   
## arch           x86_64                      
## os             darwin15.6.0                
## system         x86_64, darwin15.6.0        
## status                                     
## major          3                           
## minor          6.1                         
## year           2019                        
## month          07                          
## day            05                          
## svn rev        76782                       
## language       R                           
## version.string R version 3.6.1 (2019-07-05)
## nickname       Action of the Toes

1. Introduction

1.1 Background

Superconductors are materials that offer no resistance to electrical current. Prominent examples of superconductors include aluminium, niobium, magnesium diboride, cuprates such as yttrium barium copper oxide and iron pnictides. These materials only become superconducting at temperatures below a certain value, known as the critical temperature [nature.com]. The purpose of this project is to predict the critical tempreatures \(Tc\) of a superconductor based on a set of selected features of chemical properties.

1.2 Data Preparation

1.2.1 Libraries

The libraries that will be used in this assignment

library(caTools)
library(caret)
library(dplyr)
library(glmnet)
library(ggfortify)
library(ggplot2)
library(ggthemes)
library(gridExtra)

1.2.2 Loading Dataset

We are using the superconduct dataset from the Superconducting Material Database maintained by Japan’s National Institute for Materials Science(NIMS).

It contains 21,263 material records, each of which have 82 columns: 81 columns corresponding to the features extracted and the last 1 column of the observed Tc values. Among those 81 columns, the first column is the number of elements in the material, the rest 80 columns are features extracted from 8 properties (each property has 10 features).

# Load the data
superconductor <- read.csv("./superconduct/train.csv")
# Display the dimensions
cat("The superconductor dataset has", dim(superconductor)[1], "records, each with", dim(superconductor)[2],
    "attributes.")
## The superconductor dataset has 21263 records, each with 82 attributes.

To get an idea on how our data looks like, we called head() and tail() functions to print out the first and last few rows of the dataset.

# first and last few rows of the dataset
print(head(superconductor))
print(tail(superconductor))

To get an idea on how our target values Tc distributed in our dataset, we used summary() and hist(). We saw that the distribution is skewed right with an median of 20. All values are > 0, with a maximum at 185.

summary(superconductor$critical_temp)
##      Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
##   0.00021   5.36500  20.00000  34.42122  63.00000 185.00000
hist(superconductor$critical_temp, breaks = 80, main = "Tc", border="grey", col="dimgrey")

1.2.3 Methodology and Data Split

Now, we are going to split our data into training set/validation set/test set for model selection, fitting, and assessment, using the typical 80:10:10 ratio. Our methodology is to fit the model paremeter for any given complexity on our training set. For every fitted model, we are going to assess the performance on the validation set. We then, based on the performace, select the optimal set of tuning parameters. Finally, for that specific resulting model, we assess a notion of the generalization error using our test set.

# first we generate training set and test set
split = sample.split(superconductor$critical_temp, SplitRatio = 0.8)
training_set = subset(superconductor, split == TRUE)
test_set = subset(superconductor, split == FALSE)

# splits test set into validation set and test set
split = sample.split(test_set$critical_temp, SplitRatio = 0.5)
validation_set = subset(test_set, split == FALSE)
test_set = subset(test_set, split == FALSE)

# reveiew splitting result
split <- c("supercondictor","training_set","validation_set","test_set")
ratio <- c("100%", "80%","10%","10%")
num_records <- c(dim(superconductor)[1],dim(training_set)[1],dim(validation_set)[1],dim(test_set)[1])
num_attributes <- c(dim(superconductor)[2],dim(training_set)[2],dim(validation_set)[2],dim(test_set)[2])
data_dim <- data.frame(split, ratio,num_records, num_attributes)
data_dim
##            split ratio num_records num_attributes
## 1 supercondictor  100%       21263             82
## 2   training_set   80%       17290             82
## 3 validation_set   10%        1544             82
## 4       test_set   10%        1544             82

1.2.4 RMSE Function

The defined RMSE function below will be used for calculating RMSE for the following analysis

RMSE <- function(predicted, target) {
    se <- 0
    for (i in 1:length(predicted)) {
        se <- se + (predicted[i]-target[i])^2
    }
    return (sqrt(se/length(predicted)))
}

2. Exploratary Data Analysis

2.1. Collinearity between features

To have a general idea of our data, we first group then and generate subsuets and look at them one by one.

From the 8 plots on the correlations of the features below, we can see there are some simillars pattern across all properties. Particularly, below groups seem to always have strong positive correlations: - mean/wtd_mean/gmean - range/std/wtd_std - entropy/wtd_entropy

The relationship makes sense, as these value are derived from one another, so they all depend on each other at some point.

Subsets 1-4: 10 properties of atomatic_mass/fie/atomatic_radius/Density

# Feature 1: atomic_mass
pairs(superconductor[2:11],main = "Relationship between Properties of Atomic Mass",col="dimgrey")

# Feature 2: fie
pairs(superconductor[12:21],main="Relationship between Properties of Fie",col="dimgrey")

# Feature 3: atomic_radius
pairs(superconductor[22:31],main="Relationship between Properties of Atomic Radius",col="dimgrey")

# Feature 4: Density
pairs(superconductor[32:41], main="Relationship between Properties of Density",col="dimgrey")

Subsets 5-8: 10 properties of ElectronAffinity/FusionHeat/ThermalConductivity/Valence

# Feature 5: ElectronAffinity
pairs(superconductor[42:51],main = "Relationship between Properties of ElectronAffinity",col="dimgrey")

# Feature 6: FusionHeat
pairs(superconductor[52:61],main="Relationship between Properties of FusionHeat",col="dimgrey")

# Feature 7: ThermalConductivity
pairs(superconductor[62:71],main="Relationship between Properties of ThermalConductivity",col="dimgrey")

# Feature 8: Valence
pairs(superconductor[72:81], main="Relationship between Properties of Valence",col="dimgrey")

2.2 Correlations Between Properties

Subset 1: Mean of the 8 properties Mean values are also a good place to start, we will first plot out the correlation between pairs of all 9 variables, including number_of_elements and mean values of all 8 properties, to see if there is anything interesting between each pairs of attributes.

subset_mean <- superconductor[,c(2,12,22,32,42,52,62,72,82)]
dim(subset_mean)
## [1] 21263     9
str(subset_mean)
## 'data.frame':    21263 obs. of  9 variables:
##  $ mean_atomic_mass        : num  88.9 92.7 88.9 88.9 88.9 ...
##  $ mean_fie                : num  775 766 775 775 775 ...
##  $ mean_atomic_radius      : num  160 161 160 160 160 ...
##  $ mean_Density            : num  4654 5821 4654 4654 4654 ...
##  $ mean_ElectronAffinity   : num  81.8 90.9 81.8 81.8 81.8 ...
##  $ mean_FusionHeat         : num  6.91 7.78 6.91 6.91 6.91 ...
##  $ mean_ThermalConductivity: num  108 172 108 108 108 ...
##  $ mean_Valence            : num  2.25 2 2.25 2.25 2.25 2.25 2.25 2.25 2.25 2.25 ...
##  $ critical_temp           : num  29 26 19 22 23 23 11 33 36 31 ...

From the plot we can see there’s stronger relationship between the following pairs: - mean_fie and mean_atomic_radius - mean_atomic_mass and mean_Density - mean_atomic_radius and mean_Density

pairs(subset_mean[1:9], main="Mean of Properties",col = "dimgrey")

This subset of features tend to have similar kind of distribution of the target varialble critical_temp.

par(mfrow = c(3, 3))
hist(subset_mean$mean_atomic_mass, breaks = 20, main = "mean_atomic_mass", border="dimgrey", col="dimgrey")
hist(subset_mean$mean_fie, breaks = 20, main = "mean_fie", border="dimgrey", col="dimgrey")
hist(subset_mean$mean_atomic_radius, breaks = 20, main = "mean_atomic_radius", border="dimgrey", col="dimgrey")
hist(subset_mean$mean_Density, breaks = 20, main = "mean_Density", border="dimgrey", col="dimgrey")
hist(subset_mean$mean_ElectronAffinity, breaks = 20, main = "mean_ElectronAffinity", border="dimgrey", col="dimgrey")
hist(subset_mean$mean_FusionHeat, breaks = 20, main = "mean_FusionHeat", border="dimgrey", col="dimgrey")
hist(subset_mean$mean_ThermalConductivity, breaks = 20, main = "man_ThermalConductivity", border="dimgrey", col="dimgrey")
hist(subset_mean$mean_Valence, breaks = 20, main = "mean_Valence", border="dimgrey", col="dimgrey")
hist(subset_mean$critical_temp, breaks = 20, main = "critical_Temp", border="dimgrey", col="dimgrey")

Subset 2: entropy of the 8 properties

subset_entropy <- superconductor[,c(6,16,26,36,46,56,66,76,82)]
dim(subset_entropy)
## [1] 21263     9
str(subset_entropy)
## 'data.frame':    21263 obs. of  9 variables:
##  $ entropy_atomic_mass        : num  1.18 1.45 1.18 1.18 1.18 ...
##  $ entropy_fie                : num  1.31 1.54 1.31 1.31 1.31 ...
##  $ entropy_atomic_radius      : num  1.26 1.51 1.26 1.26 1.26 ...
##  $ entropy_Density            : num  1.03 1.31 1.03 1.03 1.03 ...
##  $ entropy_ElectronAffinity   : num  1.16 1.43 1.16 1.16 1.16 ...
##  $ entropy_FusionHeat         : num  1.09 1.37 1.09 1.09 1.09 ...
##  $ entropy_ThermalConductivity: num  0.308 0.847 0.308 0.308 0.308 ...
##  $ entropy_Valence            : num  1.37 1.56 1.37 1.37 1.37 ...
##  $ critical_temp              : num  29 26 19 22 23 23 11 33 36 31 ...
pairs(subset_entropy[1:9],main="Entropy of Properties",col="dimgrey")

We noticed this subset of features tend to have jagged and skewed distrubtion, except entropy_ThermalConductivity which is more gaussian distributed

par(mfrow = c(3, 3))
hist(subset_entropy$entropy_atomic_mass, breaks = 20, main = "entropy_atomic_mass", border="dimgrey", col="dimgrey")
hist(subset_entropy$entropy_fie, breaks = 20, main = "entropy_fie", border="dimgrey", col="dimgrey")
hist(subset_entropy$entropy_atomic_radius, breaks = 20, main = "entropy_atomic_radius", border="dimgrey", col="dimgrey")
hist(subset_entropy$entropy_Density, breaks = 20, main = "entropy_Density", border="dimgrey", col="dimgrey")
hist(subset_entropy$entropy_ElectronAffinity, breaks = 20, main = "entropy_ElectronAffinity", border="dimgrey", col="dimgrey")
hist(subset_entropy$entropy_FusionHeat, breaks = 20, main = "entropy_FusionHeat", border="dimgrey", col="dimgrey")
hist(subset_entropy$entropy_ThermalConductivity, breaks = 20, main = "entropy_ThermalConductivity", border="dimgrey", col="dimgrey")
hist(subset_entropy$entropy_Valence, breaks = 20, main = "entropy_Valence", border="dimgrey", col="dimgrey")
hist(subset_entropy$critical_temp, breaks = 20, main = "critical_Temp", border="dimgrey", col="dimgrey")

Subset 3: std of the 8 properties

subset_std <- superconductor[,c(10,20,30,40,50,60,70,80,82)]
dim(subset_std)
## [1] 21263     9
str(subset_std)
## 'data.frame':    21263 obs. of  9 variables:
##  $ std_atomic_mass        : num  52 47.1 52 52 52 ...
##  $ std_fie                : num  324 290 324 324 324 ...
##  $ std_atomic_radius      : num  75.2 67.3 75.2 75.2 75.2 ...
##  $ std_Density            : num  3306 3767 3306 3306 3306 ...
##  $ std_ElectronAffinity   : num  51.4 49.4 51.4 51.4 51.4 ...
##  $ std_FusionHeat         : num  4.6 4.47 4.6 4.6 4.6 ...
##  $ std_ThermalConductivity: num  169 199 169 169 169 ...
##  $ std_Valence            : num  0.433 0.632 0.433 0.433 0.433 ...
##  $ critical_temp          : num  29 26 19 22 23 23 11 33 36 31 ...
pairs(subset_std[1:9],main="Standard Deviation of Properties",col="dimgrey")

Interestingly, this subset of features tend to have multimodal distribution.

par(mfrow = c(3, 3))
hist(subset_std$std_atomic_mass, breaks = 20, main = "std_atomic_mass", border="dimgrey", col="dimgrey")
hist(subset_std$std_fie, breaks = 20, main = "std_fie", border="dimgrey", col="dimgrey")
hist(subset_std$std_atomic_radius, breaks = 20, main = "std_atomic_radius", border="dimgrey", col="dimgrey")
hist(subset_std$std_Density, breaks = 20, main = "std_Density", border="dimgrey", col="dimgrey")
hist(subset_std$std_ElectronAffinity, breaks = 20, main = "std_ElectronAffinity", border="dimgrey", col="dimgrey")
hist(subset_std$std_FusionHeat, breaks = 20, main = "std_FusionHeat", border="dimgrey", col="dimgrey")
hist(subset_std$std_ThermalConductivity, breaks = 20, main = "std_ThermalConductivity", border="dimgrey", col="dimgrey")
hist(subset_std$std_Valence, breaks = 20, main = "std_Valence", border="dimgrey", col="dimgrey")
hist(subset_std$critical_temp, breaks = 20, main = "critical_Temp", border="dimgrey", col="dimgrey")

2.3 Principle Component Analysis

In statistics, PCA is an unsupervised method that linearly projects data from a high dimensional space into a lower dimensional space. By maximising the variance of each of the new, uncorrelated dimensions (principal components), we are able to extract most of the underlying structure and relationships inherent to the original raw data.

Now, because we have observed strong multicollinearity in our data, we are now going to try to use PCA as a tool to better understand and visualise the variance in our dataset in lower dimensions.

# Principal Component 
pca_model <- prcomp(training_set[,c(1:81)], center = TRUE,scale. = TRUE)
summary(pca_model)
## Importance of components:
##                           PC1    PC2     PC3     PC4     PC5    PC6     PC7
## Standard deviation     5.6123 2.9097 2.77638 2.53431 2.19071 1.7498 1.70915
## Proportion of Variance 0.3889 0.1045 0.09516 0.07929 0.05925 0.0378 0.03606
## Cumulative Proportion  0.3889 0.4934 0.58855 0.66785 0.72710 0.7649 0.80096
##                            PC8     PC9    PC10    PC11    PC12    PC13    PC14
## Standard deviation     1.59384 1.38788 1.25962 1.21830 1.09061 0.98034 0.89741
## Proportion of Variance 0.03136 0.02378 0.01959 0.01832 0.01468 0.01187 0.00994
## Cumulative Proportion  0.83232 0.85610 0.87569 0.89402 0.90870 0.92056 0.93051
##                           PC15    PC16    PC17    PC18    PC19    PC20    PC21
## Standard deviation     0.89414 0.79692 0.75704 0.66623 0.62741 0.55123 0.49471
## Proportion of Variance 0.00987 0.00784 0.00708 0.00548 0.00486 0.00375 0.00302
## Cumulative Proportion  0.94038 0.94822 0.95529 0.96077 0.96563 0.96938 0.97241
##                          PC22    PC23    PC24    PC25    PC26    PC27    PC28
## Standard deviation     0.4847 0.45592 0.40849 0.40012 0.38704 0.36836 0.33874
## Proportion of Variance 0.0029 0.00257 0.00206 0.00198 0.00185 0.00168 0.00142
## Cumulative Proportion  0.9753 0.97787 0.97993 0.98191 0.98376 0.98543 0.98685
##                           PC29    PC30    PC31    PC32    PC33    PC34    PC35
## Standard deviation     0.32001 0.30682 0.28763 0.27815 0.27542 0.24097 0.23694
## Proportion of Variance 0.00126 0.00116 0.00102 0.00096 0.00094 0.00072 0.00069
## Cumulative Proportion  0.98811 0.98928 0.99030 0.99125 0.99219 0.99291 0.99360
##                           PC36    PC37    PC38    PC39    PC40    PC41    PC42
## Standard deviation     0.22401 0.21451 0.19797 0.18718 0.18474 0.16197 0.15789
## Proportion of Variance 0.00062 0.00057 0.00048 0.00043 0.00042 0.00032 0.00031
## Cumulative Proportion  0.99422 0.99479 0.99527 0.99570 0.99612 0.99645 0.99676
##                           PC43    PC44    PC45    PC46   PC47    PC48    PC49
## Standard deviation     0.14466 0.13867 0.13372 0.13220 0.1267 0.12400 0.12103
## Proportion of Variance 0.00026 0.00024 0.00022 0.00022 0.0002 0.00019 0.00018
## Cumulative Proportion  0.99701 0.99725 0.99747 0.99769 0.9979 0.99808 0.99826
##                           PC50    PC51    PC52    PC53    PC54    PC55    PC56
## Standard deviation     0.11918 0.11260 0.11078 0.10123 0.09882 0.09732 0.09186
## Proportion of Variance 0.00018 0.00016 0.00015 0.00013 0.00012 0.00012 0.00010
## Cumulative Proportion  0.99843 0.99859 0.99874 0.99887 0.99899 0.99910 0.99921
##                           PC57    PC58    PC59    PC60    PC61    PC62    PC63
## Standard deviation     0.08503 0.08104 0.08027 0.07608 0.07274 0.06782 0.05995
## Proportion of Variance 0.00009 0.00008 0.00008 0.00007 0.00007 0.00006 0.00004
## Cumulative Proportion  0.99930 0.99938 0.99946 0.99953 0.99960 0.99965 0.99970
##                           PC64    PC65    PC66    PC67    PC68    PC69    PC70
## Standard deviation     0.05960 0.05639 0.05339 0.05090 0.04760 0.04296 0.04072
## Proportion of Variance 0.00004 0.00004 0.00004 0.00003 0.00003 0.00002 0.00002
## Cumulative Proportion  0.99974 0.99978 0.99982 0.99985 0.99988 0.99990 0.99992
##                           PC71    PC72    PC73    PC74    PC75    PC76    PC77
## Standard deviation     0.03846 0.03683 0.03458 0.02784 0.02489 0.02089 0.01812
## Proportion of Variance 0.00002 0.00002 0.00001 0.00001 0.00001 0.00001 0.00000
## Cumulative Proportion  0.99994 0.99995 0.99997 0.99998 0.99999 0.99999 0.99999
##                           PC78    PC79     PC80    PC81
## Standard deviation     0.01362 0.01089 0.008585 0.00699
## Proportion of Variance 0.00000 0.00000 0.000000 0.00000
## Cumulative Proportion  1.00000 1.00000 1.000000 1.00000

The result tells us that we are actually able to capture up to almost 99% of variance in the entire dataset using only 30 principal components.

Now we will quickly run a Principal Component Regression using some of the key principal components we just calculated and see how it goes.

pcr_model <- train(critical_temp ~ .,
                         data = training_set,
                         method = 'pcr',
                         tuneGrid = expand.grid(ncomp = seq(2,40,2)),
                         trControl = ,
                         preProc = c('center','scale','BoxCox'))


pcr_model$results
##    ncomp     RMSE  Rsquared      MAE    RMSESD  RsquaredSD     MAESD
## 1      2 25.91598 0.4340713 21.12392 0.1870371 0.004371131 0.1352689
## 2      4 23.59378 0.5309428 18.75033 0.1613195 0.004617488 0.1195303
## 3      6 22.25655 0.5825855 17.70229 0.1204943 0.005058058 0.1317304
## 4      8 22.23655 0.5833334 17.59996 0.1227173 0.004968249 0.1066751
## 5     10 22.23127 0.5835297 17.57764 0.1234933 0.004984291 0.1083197
## 6     12 21.90477 0.5956759 17.26656 0.1266269 0.005641423 0.1005386
## 7     14 21.78913 0.5999365 17.14338 0.1259035 0.005517950 0.1016760
## 8     16 21.57609 0.6077168 17.03000 0.1231018 0.005751325 0.1007561
## 9     18 21.44371 0.6125268 16.89100 0.1244964 0.005549185 0.1027890
## 10    20 21.28112 0.6183859 16.83165 0.1214638 0.005731132 0.1062709
## 11    22 20.89875 0.6319685 16.71036 0.1268960 0.005834470 0.1061440
## 12    24 20.58353 0.6429930 16.34391 0.1530739 0.006460945 0.1100342
## 13    26 19.79650 0.6697523 15.58452 0.1639544 0.007013933 0.1201760
## 14    28 19.77439 0.6704892 15.57147 0.1519972 0.006749447 0.1188111
## 15    30 19.51542 0.6790511 15.23949 0.1715930 0.007098828 0.1622978
## 16    32 19.36942 0.6838576 15.02386 0.1792539 0.006888911 0.1141317
## 17    34 19.36122 0.6841234 15.01951 0.1863832 0.007303873 0.1179752
## 18    36 19.30942 0.6858202 14.95558 0.1742180 0.006772224 0.1316568
## 19    38 18.92883 0.6980752 14.50738 0.1655878 0.006702987 0.1187027
## 20    40 18.94355 0.6976208 14.50319 0.1827815 0.007312864 0.1160223
pcr_model$bestTune
##    ncomp
## 19    38

Not too well with just an rsqured of 0.7 wiht 40 principal componets. The underfitted model might be the result of a small number of principal components \(d\), or the potential non-linear relationship between the predictors and response variable. Which we will be discussing more in the model selection section.

But what are the significant features identified by this algorithm?

pcr_features = varImp(pcr_model)

pcr_top40 = data.frame(feature = pcr_features$importance%>% rownames(),
           overall = pcr_features$importance$Overall)

pcr_top40 = pcr_top40[order(pcr_top40$overall,decreasing = TRUE),][1:40,]

# Generates a slice for top 40 important features
pcrFeatures = pcr_top40$feature %>% as.character()

# generates a subset from superconductor
pcrFeatures <- superconductor %>% select(append(pcrFeatures,"critical_temp"))

We plot the histograms of all these features. We noticed that there are 4 features having similar distributions with critical_temp, including:

  • wtd_gmean_Density
  • wtd_mean_FusionHeat
  • wtd_gmean_FusionHeat
  • wtd_range_FusionHeat
# log transformation on features with skewed dist
N = ncol(pcrFeatures)
colorcode <- rep("dimgrey",N)
colorcode[N] <- "deeppink"
par(mfrow=c(3, 3))
for (i in 1:(N)) {
  hist(pcrFeatures[,i], breaks = 20, main = paste(i,names(pcrFeatures)[i],sep="."), border=colorcode[i],
       col=colorcode[i],cex.main=0.9)
}

We further explore some interesting relationships between the significant features through data visualisation.

ggplot(superconductor, 
       aes(x =wtd_gmean_Density , y =critical_temp, color = wtd_std_ThermalConductivity)) +
  geom_point(aes(size =number_of_elements), alpha = 0.4) +
  ggtitle("Data Exploration - Figure 1") +
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_color_distiller(palette = "Paired") +theme_light()

ggplot(superconductor, 
       aes(x = number_of_elements , y = wtd_gmean_Valence, color = wtd_mean_Valence)) +
  geom_point(aes(size = wtd_std_ThermalConductivity), alpha = 0.4) +
  ggtitle("Data Exploration - Figure 2") +
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_color_distiller(palette = "Paired") +theme_light()

ggplot(superconductor, 
       aes(x = number_of_elements , y =wtd_gmean_Density , color = entropy_atomic_mass)) +
  geom_point(aes(size = wtd_std_ThermalConductivity), alpha = 0.4) +
  ggtitle("Data Exploration - Figure 3") +
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_color_distiller(palette = "Paired") +theme_light()

ggplot(superconductor, 
       aes(x = number_of_elements , y =entropy_atomic_mass, color =critical_temp )) +
  geom_point(aes(size = mean_atomic_mass), alpha = 0.4) +
  ggtitle("Data Exploration - Figure 4") +
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_color_distiller(palette = "Paired") +theme_light()

ggplot(superconductor, 
       aes(x = wtd_entropy_Valence , y =wtd_entropy_atomic_radius , color = wtd_entropy_FusionHeat)) +
  geom_point(aes(size = range_atomic_radius), alpha = 0.4) +
  ggtitle("Data Exploration - Figure 5") +
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_color_distiller(palette = "Paired") +theme_light()

ggplot(superconductor, 
       aes(x = wtd_gmean_Density, y =wtd_gmean_fie , color = wtd_std_fie)) +
  geom_point(aes(size = range_fie), alpha = 0.4) +
  ggtitle("Data Exploration - Figure 6") +
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_color_distiller(palette = "Paired") +theme_light()

library(ggthemes)
ggplot(superconductor, 
       aes(x = wtd_gmean_Density, y =log(critical_temp), color =wtd_std_atomic_radius)) +
  geom_point(aes(size = std_atomic_radius), alpha = 0.4) +
  ggtitle("Data Exploration - Figure 7") +
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_color_distiller(palette = "Paired")  +theme_light()

ggplot(superconductor, 
       aes(x = range_fie, y =std_atomic_radius, color =wtd_std_atomic_radius)) +
  geom_point(aes(size = wtd_mean_FusionHeat), alpha = 0.4) +
  ggtitle("Data Exploration - Figure 8") +
  theme(plot.title = element_text(hjust = 0.5)) +
  scale_color_distiller(palette = "Paired") +theme_light()

3. Model Developmemnt

3.1 Linear regression

3.1.1 Linear regression with all variables

Our first model is going to be the simplest one, fitting a linear regression model to all of the 81 indicators. We fit the model by calling lm(), and then we call summary() to summarize the results.

# Fitting Simple Linear Regression to the Training set
fit1.1 = lm(formula = critical_temp ~ .,
               data = training_set)

There are a few points we wanted to highlight here: - The R-squred 0.74 indicates that the model explains 74% of the variation in Tc - The F-statistic 608 has a p-value < 2.2e-16, so reject the null hypothesis (the model explains nothing) and accept the althernative hypothesis that the model is useful

# prints important stats
num_features1.1 <- dim(summary(fit1.1)$coefficients)[1]-1
cat("Number of features in model 1.1 = ",num_features1.1)
## Number of features in model 1.1 =  81
cat("\nRsquared = ",summary(fit1.1)$adj.r.squared)
## 
## Rsquared =  0.7394145
cat("\nF-statistic =", summary(fit1.1)$fstatistic[1])
## 
## F-statistic = 606.6513

3.1.1 Linear Regression with Stepwise Feature Selection

Initially, we had all 81 in our model. But we doubted if they are all that important and necessary. To really find out the optmized subset of features, the all subset algorithms is an option. However, in this case, given the size of our dataset and the number of attibutes in hand, due to the fact that the computational complexity of such brute force algorithm is exponential.

Our other option is to do a sub-optimal approach such as stepwise algorithm for feature selection. How stepwise search (or Greedy search) works is you start with some set of possible features (or zero feature), and then you greedily walk through features, and select the best one to take or drop, and then you keep iterating. Here, we ran the step() function, a stpewise algorithm for feature selection by AIC, to find it out.

Even though this procedure is significantly more computational efficient

O(\(D^{2}\)) >> O(\(2^{D}\)) for large D

given the search data size and search time, we only did backwards and both direcation searches this time.

# Run step to remove unnecessary variables
sback_fit1.1 = step(fit1.1,direction = "backward")
sboth_fit1.1 = step(fit1.1,direction = "both")
# extract AIC
aic_fit1.1 <- extractAIC(fit1.1)
aic_sback_fit1.1 <- extractAIC(sback_fit1.1)
aic_sboth_fit1.1 <- extractAIC(sboth_fit1.1)

According to the step() results, the backward/both selection gave us the same results, removing 11 features from our model and achieved lower AIC.

stepResults.fit1 = data.frame(
  "num_predictors" = 
    c("beginning.fit"=aic_fit1.1[1]-1,
                      "step.backward"=aic_sback_fit1.1[1]-1,"step.both"=aic_sboth_fit1.1[1]-1),
  "aic" = 
    c("beginning.fit"=aic_fit1.1[2],
             "step.backward"=aic_sback_fit1.1[2],"step.both"=aic_sboth_fit1.1[2])
)
stepResults.fit1
##               num_predictors      aic
## beginning.fit             81 99149.91
## step.backward             73 99140.21
## step.both                 73 99140.21

Now let’s prepare to remove these features and update the model

# feature removed by step()
removed <- sback_fit1.1$anova$Step
# string argument for updating the linear model
formula = paste(".~.",paste(removed,collapse = ""),sep = "")
formula
## [1] ".~.- wtd_range_Density- wtd_std_ThermalConductivity- wtd_mean_Density- wtd_range_Valence- wtd_range_atomic_mass- wtd_entropy_atomic_mass- wtd_entropy_ThermalConductivity- gmean_atomic_radius"

We updated our model 1.1 and removed the 7 features by using the string input we defined above. The updated model gave us a similar rsqured while using less predictors.

# update fit1
fit1.2 <- update(fit1.1,formula)

# prints out stats
num_features1.2 <- dim(summary(fit1.2)$coefficients)[1]-1
cat("Number of features in model 1.2 = ",num_features1.2)
## Number of features in model 1.2 =  73
cat("\nRsquared = ",summary(fit1.2)$adj.r.squared)
## 
## Rsquared =  0.7394408

3.1.2 Linear Regression with Significant Variables

In addition to the step() function above, we wanted to explore some other options in feature selection. Here, we used varImp() function to calculate importance of all 81 predictors initially in our first model.

# train the model
fit1.3 <- train(critical_temp ~., data = training_set, method = 'lm',preProcess="scale",trControl = trainControl(method = "cv"))

# List of features with their importance scores
importance1.3 <- varImp(fit1.3, scale=FALSE)
print(importance1.3)
## lm variable importance
## 
##   only 20 most important variables shown (out of 81)
## 
##                               Overall
## std_ElectronAffinity           18.657
## range_ElectronAffinity         18.390
## wtd_mean_ThermalConductivity   16.263
## wtd_mean_atomic_radius         12.905
## wtd_entropy_FusionHeat         12.432
## wtd_std_ElectronAffinity       12.336
## wtd_range_ThermalConductivity  12.238
## wtd_gmean_atomic_radius        11.654
## wtd_gmean_ElectronAffinity     11.593
## range_atomic_mass              11.526
## wtd_std_Valence                11.127
## wtd_entropy_Valence            10.655
## wtd_gmean_ThermalConductivity  10.402
## range_fie                       9.652
## wtd_mean_FusionHeat             9.580
## wtd_mean_ElectronAffinity       9.263
## mean_atomic_mass                9.129
## wtd_entropy_ElectronAffinity    9.117
## wtd_gmean_FusionHeat            8.593
## mean_FusionHeat                 8.593

Let’s organize the data nad rank the features by importance score and extract the top 60 for modeling

# extracts reletive elements from the original output list
rank = importance1.3$importance$Overall

# generates a new list called features including columns of feature names and their rankings
features <- training_set %>% select(-critical_temp) %>% names()
important1.3 = data.frame(features, rank)
important1.3 = important1.3[order(important1.3$rank,decreasing=TRUE),]

# Generates a slice for top 60 important features
top60 = important1.3[1:60,1] %>% as.character()
top60
##  [1] "std_ElectronAffinity"          "range_ElectronAffinity"       
##  [3] "wtd_mean_ThermalConductivity"  "wtd_mean_atomic_radius"       
##  [5] "wtd_entropy_FusionHeat"        "wtd_std_ElectronAffinity"     
##  [7] "wtd_range_ThermalConductivity" "wtd_gmean_atomic_radius"      
##  [9] "wtd_gmean_ElectronAffinity"    "range_atomic_mass"            
## [11] "wtd_std_Valence"               "wtd_entropy_Valence"          
## [13] "wtd_gmean_ThermalConductivity" "range_fie"                    
## [15] "wtd_mean_FusionHeat"           "wtd_mean_ElectronAffinity"    
## [17] "mean_atomic_mass"              "wtd_entropy_ElectronAffinity" 
## [19] "wtd_gmean_FusionHeat"          "mean_FusionHeat"              
## [21] "gmean_FusionHeat"              "mean_Density"                 
## [23] "wtd_range_FusionHeat"          "std_atomic_mass"              
## [25] "range_atomic_radius"           "std_Density"                  
## [27] "wtd_mean_atomic_mass"          "entropy_atomic_mass"          
## [29] "wtd_entropy_fie"               "std_fie"                      
## [31] "wtd_entropy_Density"           "wtd_entropy_atomic_radius"    
## [33] "entropy_FusionHeat"            "wtd_range_ElectronAffinity"   
## [35] "std_ThermalConductivity"       "range_ThermalConductivity"    
## [37] "range_Density"                 "wtd_gmean_atomic_mass"        
## [39] "entropy_fie"                   "gmean_atomic_mass"            
## [41] "wtd_range_fie"                 "entropy_Valence"              
## [43] "wtd_range_atomic_radius"       "entropy_ThermalConductivity"  
## [45] "wtd_std_FusionHeat"            "range_Valence"                
## [47] "entropy_Density"               "entropy_atomic_radius"        
## [49] "range_FusionHeat"              "number_of_elements"           
## [51] "wtd_std_atomic_radius"         "gmean_ElectronAffinity"       
## [53] "wtd_gmean_Density"             "wtd_gmean_Valence"            
## [55] "wtd_std_Density"               "mean_atomic_radius"           
## [57] "std_atomic_radius"             "gmean_Valence"                
## [59] "wtd_mean_Valence"              "std_Valence"

Re-fits Top60 Features

This time, with using only 60 selected features, we achieve almost the same r-square 0.74 we obtained using up to 74 features previously. In this case, we would say thay the varImp() did better job than step() in feature selection, because it provides more effiecient solution for our model.

# generates a subset of traninig data for top 60 features
features_top60 <- append(top60,"critical_temp")
training1.3 <- training_set %>% select(features_top60)
## Note: Using an external vector in selections is ambiguous.
## ℹ Use `all_of(features_top60)` instead of `features_top60` to silence this message.
## ℹ See <https://tidyselect.r-lib.org/reference/faq-external-vector.html>.
## This message is displayed once per session.
# refits model using top60 features
fit1.3 = lm(formula = critical_temp~ .,
               data = training1.3)

num_features1.3 <- dim(summary(fit1.3)$coefficients)[1]-1
cat("Number of features in model 1.3 = ",num_features1.3)
## Number of features in model 1.3 =  60
cat("\nRsquared = ",summary(fit1.3)$adj.r.squared)
## 
## Rsquared =  0.7371022

Assessment on Validation Set

Now that we have three fitted models at the moment, we are going to assess each of them on the validation set and compare their performance.

# Model 1.1 Predicting Tc for training/validation set 
pred_train1.1 = predict(fit1.1, newdata = training_set)
rmse_train1.1 <- RMSE(pred_train1.1, training_set$critical_temp)
pred_validation1.1 = predict(fit1.1, newdata = validation_set)
rmse_v1.1 <- RMSE(pred_validation1.1, validation_set$critical_temp)
rsq_v1.1 <- cor(pred_validation1.1, validation_set$critical_temp)^2

# Model 1.2 Predicting Tc for training/validation set
pred_train1.2 = predict(fit1.2, newdata = training_set)
rmse_train1.2 <- RMSE(pred_train1.2, training_set$critical_temp)
pred_validation1.2 = predict(fit1.2, newdata = validation_set)
rmse_v1.2 <- RMSE(pred_validation1.2, validation_set$critical_temp)
rsq_v1.2 <- cor(pred_validation1.2, validation_set$critical_temp)^2

# Model 1.3 Predicting Tc for training/validation set
pred_train1.3 = predict(fit1.3, newdata = training_set)
rmse_train1.3 <- RMSE(pred_train1.3, training_set$critical_temp)
pred_validation1.3 = predict(fit1.3, newdata = validation_set)
rmse_v1.3 <- RMSE(pred_validation1.3, validation_set$critical_temp)
rsq_v1.3 <- cor(pred_validation1.3, validation_set$critical_temp)^2

RMSE Analysis

Comparing the three models, we believe that model 1.3 is a better one, since it was able to acheieve the similar results using less predictors.

lin_reg_model1.1 <- c("num_predictors"=num_features1.1,
             "adj.rsquared_train"=summary(fit1.1)$adj.r.squared,
             "adj.rsquared_validation"=rsq_v1.1,
             "rmse_validation"=rmse_v1.1)

lin_reg_model1.2 <- c("num_predictors"=num_features1.2,
           "adj.rsquared_train"=summary(fit1.2)$adj.r.squared,
           "adj.rsquared_validation"=rsq_v1.2,
           "rmse_validation"=rmse_v1.2)

lin_reg_model1.3 <- c("num_predictors"=num_features1.3,
           "adj.rsquared_train"=summary(fit1.3)$adj.r.squared,
           "adj.rsquared_validation"= rsq_v1.3,
           "rmse_validation"= rmse_v1.3)

# creates a df by combing above vectors
models.1 <- data.frame(lin_reg_model1.1,lin_reg_model1.2,lin_reg_model1.3)
models.1
##                         lin_reg_model1.1 lin_reg_model1.2 lin_reg_model1.3
## num_predictors                81.0000000       73.0000000       60.0000000
## adj.rsquared_train             0.7394145        0.7394408        0.7371022
## adj.rsquared_validation        0.6963862        0.6964549        0.6951216
## rmse_validation.3             18.4305527       18.4284397       18.4647374

Diagnostic Plots

We will firt call autoplot() from ggfortify package to calculate and produce diagnostic plots and see if there are any serious problems inherented in our model.

  • Residuals vs Fitted : shows the residuals are not evenly distributed around zero, this suggests that the assumption that the relationship is linear is not reasonable, also the variances of the error terms are unequal.
  • Normal Q-Q indicates the plot tells us there’s evidence of non-linearity
  • Scale-Location plot tells us the residuals are not spread equally along the ranges of predictors
  • Residuals vs Leverage helps us to find influential cases, but there seems no such case here
autoplot(fit1.3)

Generalization of Error

Since model 1.3 has been selected as our first official model based on the performance on the validation, we are now going to assess its performance on the test set. We then assume the test errors as an approximation of our generalization error.

# Predicting Tc for training/test
pred_test1.3 = predict(fit1.3, newdata = test_set)
rmse_test1.3 <- RMSE(pred_test1.3,test_set$critical_temp)
# Calculates RMSE of training pred
cat("\nLINEAR REGRESSION MODEL 1.3: RMSE for the test predictions =", rmse_test1.3)
## 
## LINEAR REGRESSION MODEL 1.3: RMSE for the test predictions = 18.46474

Visualization of Gernerailzed Error

As we can see, we got this very similar RMSE that we have seen on the training/validation set. To get a better idea on the fitness, we visualized the perfomance using ggplot().

From the plot, we could tell that, althoght not that strong, there’s still a linear relationship between the true values and the predicted ones. That should be a fair representation on the fitness of this model.

# Visualizing the fit
Linear_regression_test <- ggplot() +
  geom_point(aes(x = test_set$critical_temp, y = pred_test1.3),
            colour = 'gold2',alpha=0.5,size=3) +
  ggtitle('Linear Regression') +
  ylab('Prediction') +
  xlab('True Value (Tc)') +
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

Linear_regression_test 

3.1.3 Linear Regression with Transfored Dataset

To further improve our model, we are going to try and generate two-way interaction terms for fitting our second model. Particularly, we are going to generate interaction terms using features with low collinearity, since we know that collinearity, which is the correlation between predictor variables supply redundant information to the model, and may consequently effect model performance.

Thus, the first step is to pin down these features using the findCorrelation() function. By setting the cutoff threshold as 0.8, we got 29 selected features

Collinearity Analysis

# Identifies highly correlated terms
correlationMatrix <- cor(training_set[,1:81])

# findCorrelation() searches through a correlation matrix 
# and returns a vector of integers corresponding to columns to remove to reduce pair-wise correlations.
highlyCorrelated <- findCorrelation(correlationMatrix, cutoff=0.8)
cat("Number of highly correlated features (to be removed) =",length(highlyCorrelated))
## Number of highly correlated features (to be removed) = 52
# creates a subset containing only features with correlation < 0.8
lowCor = training_set[,-highlyCorrelated]
cat("\nNumber of features with correlation lower than 0.8 (to be selected) =",length(lowCor)-1)
## 
## Number of features with correlation lower than 0.8 (to be selected) = 29

Skewness Analysis

As we looked at the histograms of all 29 features selected based on the collinearity, we found that most of the features have weird distributions.

# looks at distribution of each feature
par(mfrow=c(3, 3))
N = ncol(lowCor) -1
for (i in 1:N) {
  hist(lowCor[,i], breaks = 20, main = paste(i,names(lowCor)[i],sep = ". "), border="grey", col="darkgrey")
}

Account for the Heteroscedasticity

Since there are too many features to look at at a time, we automatically generated histograms for those features (those do not have any zero values) with log transformation, and check the distributions again of each one of them.

We spotted the three of them (highlighted in yellow) have become more gaussian in their distributions after log transformation. These following five features below with log transformations will be included in our model:

  • gmean_fie
  • mean_Density
  • gmean_ElectronAffinity
  • wtd_gmean_ElectronAffinity
  • mean_FusionHeat
# log transformation on features with skewed dist
par(mfrow=c(3, 3))
N = ncol(lowCor) -1
colorcode <- rep("gray87",29)
colorcode[c(3,9,13,14,18)] <- "goldenrod"
for (i in 1:N) {
  if (min(lowCor[,i])>0){
    hist(log(lowCor[,i]), breaks = 20, main = paste(i,paste("log(",names(lowCor)[i],")",sep = ""),sep = ". "), border=colorcode[i], col=colorcode[i])
  }
}

We removed the three features with skewed distributions and added the log transformed terms back into our string argument for fitting the model

# generate a subset of features with low correlation coeficients, while excluding 3 highly skewed terms
features2.1 <- lowCor %>% select(-c(3,9,13,14,18))%>%select(-critical_temp) %>% names()

# adding back the three originally skewed terms after applying log transformation, and generates an string argument for updating the model fit
formula <- paste(paste(features2.1,collapse = "+"),"log(gmean_fie) + log(mean_Density) + log(gmean_ElectronAffinity)+log(wtd_gmean_ElectronAffinity)+log(mean_FusionHeat))^2",sep = "+")
formula <- paste0(".~. +(",formula)
formula
## [1] ".~. +(mean_atomic_mass+std_atomic_mass+wtd_gmean_fie+wtd_range_fie+mean_atomic_radius+wtd_range_atomic_radius+std_atomic_radius+wtd_entropy_Density+wtd_range_Density+wtd_std_Density+wtd_entropy_ElectronAffinity+wtd_range_ElectronAffinity+std_ElectronAffinity+wtd_range_FusionHeat+std_FusionHeat+mean_ThermalConductivity+wtd_gmean_ThermalConductivity+entropy_ThermalConductivity+wtd_entropy_ThermalConductivity+wtd_range_ThermalConductivity+std_ThermalConductivity+gmean_Valence+range_Valence+wtd_range_Valence+log(gmean_fie) + log(mean_Density) + log(gmean_ElectronAffinity)+log(wtd_gmean_ElectronAffinity)+log(mean_FusionHeat))^2"

Fit the interaction terms (feature crosses) to the model

The rsquare value turned out to be 0.82, much higher than the previous 0.74. However, we have also seen a big jump in the number of features. Maybe some of the features are not that important and thus can be excluded. To reduce the number of features while trying to keep up with the good performance, we are going to do feature selection later.

# refits the model by updating model 1
fit2.1 <- update(fit1.3,formula,data=training_set)

# prints stats to console
num_features2.1 <- dim(summary(fit2.1)$coefficients)[1]-1
cat("MODEL 2.1: number of predictors = ",num_features2.1)
## MODEL 2.1: number of predictors =  477
cat("\nTraining adj.rsquared = ",summary(fit2.1)$adj.r.squared)
## 
## Training adj.rsquared =  0.8152777

Diagnostic plots

  • Residuals vs Fitted plot tells us the residuals have non-linear patterns
  • Normal Q-Q indicates the residuals are not normally distributed.
  • Scale-Location plot suggests that the residuals are not spread equally along the ranges of predictors, as the residuals do not appear randomly spread
  • Residuals vs Leverage gives us a typical look when there is no influential case
autoplot(fit2.1)

Performance of model 2.1 on the validation set was better than our previous models, as we got a lower RMSE here

# Predicting Tc for validation set
pred_v2.1 <- predict(fit2.1, newdata = validation_set)

# Calculates RMSE
rmse_v2.1 <- RMSE(pred_v2.1, validation_set$critical_temp)
cat("\nMODEL 2.1: Validation RMSE =", rmse_v2.1)
## 
## MODEL 2.1: Validation RMSE = 16.3187
# Rsquared
rsq_v2.1 <- cor(pred_v2.1, validation_set$critical_temp)^2
cat("\nValidation adj.rsquared = ",rsq_v2.1)
## 
## Validation adj.rsquared =  0.761883

3.1.4 Linear Regression with Regularisation

As more features we use, the more complex the model becomes, and more likely it becomes overfit. And high complexity models could have low bias, but high variance. In this case, we want to trade off between bias and variance to get to that sweet spot of having good predictive performance.

One way to automatically balance between bias and variance is called regularization. To balance between the two measures, we introduced a new term \(lambda\) and modified the cost function as below:

Total Cost = (Measure of Fit) + \(\lambda\)*(Magnitude of Coefficients)

In essence, the tuning parameter \(\lambda\) controls model complexity, and controls such bias/variance trade-off.

In this section, we are going to use two regulariztion techniques: Ridge Regression and Lasso Regression to fit to our second model.

Data preparation

When we ran the lm() function on the interaction terms, we input this string argument critical_temp ~(.)^2. But it does not work for the glmnet function, as takes matices as input arguments. So we need to manually create the set of features.

To generate the interaction terms, the first step is to put together all the selected features (with low collinearity) as a matrix using model.matrix(). After the interaction terms are generated , we convert the matrix back to dataframe, so we can easily combine the top 60 important features used in model 1 and the interaction terms as the training set for regularization.

# creates a copy of 60 important features used in model 1 
training2.2 <- training1.3
validation2.2 <- validation_set %>% select(names(training2.2))
test2.2 <- test_set  %>% select(names(training2.2))

# generates subset of features with low correlation while replaces the originally skewed terms with log transformed terms
# generates string argument for later fitting the model.matrix()
features_lowCor <- lowCor %>% select(-c(3,9,13))%>%select(-critical_temp) %>% names()
formula <- paste("critical_temp~(",paste(paste(features_lowCor,collapse = "+"),"log(gmean_fie) + log(mean_Density) + log(gmean_ElectronAffinity))^2",sep = "+"),sep="")
formula
## [1] "critical_temp~(mean_atomic_mass+std_atomic_mass+wtd_gmean_fie+wtd_range_fie+mean_atomic_radius+wtd_range_atomic_radius+std_atomic_radius+wtd_entropy_Density+wtd_range_Density+wtd_std_Density+wtd_gmean_ElectronAffinity+wtd_entropy_ElectronAffinity+wtd_range_ElectronAffinity+std_ElectronAffinity+mean_FusionHeat+wtd_range_FusionHeat+std_FusionHeat+mean_ThermalConductivity+wtd_gmean_ThermalConductivity+entropy_ThermalConductivity+wtd_entropy_ThermalConductivity+wtd_range_ThermalConductivity+std_ThermalConductivity+gmean_Valence+range_Valence+wtd_range_Valence+log(gmean_fie) + log(mean_Density) + log(gmean_ElectronAffinity))^2"

Prepares training set for regularization…

# creates a matrix for interaction terms using the string argument we created above
train_interact <- model.matrix(critical_temp~(mean_atomic_mass+std_atomic_mass+wtd_gmean_fie+wtd_range_fie+mean_atomic_radius+wtd_range_atomic_radius+std_atomic_radius+wtd_entropy_Density+wtd_range_Density+wtd_std_Density+wtd_gmean_ElectronAffinity+wtd_entropy_ElectronAffinity+wtd_range_ElectronAffinity+std_ElectronAffinity+mean_FusionHeat+wtd_range_FusionHeat+std_FusionHeat+mean_ThermalConductivity+wtd_gmean_ThermalConductivity+entropy_ThermalConductivity+wtd_entropy_ThermalConductivity+wtd_range_ThermalConductivity+std_ThermalConductivity+gmean_Valence+range_Valence+wtd_range_Valence+log(gmean_fie) + log(mean_Density) + log(gmean_ElectronAffinity))^2,training_set)[,-1]

# removes duplicates by indexing and filtering
train_interact <- train_interact %>% as.data.frame()
train_interact <- train_interact[which(!names(train_interact) %in% names(training2.2))]
training2.2 <- cbind(train_interact,training2.2)
dim(training2.2)
## [1] 17290   476

Same procedure of preparing validation set for regulariztion…

# creates a set of features including interaction terms (test set)
v_interact <- model.matrix(critical_temp~(mean_atomic_mass+std_atomic_mass+wtd_gmean_fie+wtd_range_fie+mean_atomic_radius+wtd_range_atomic_radius+std_atomic_radius+wtd_entropy_Density+wtd_range_Density+wtd_std_Density+wtd_gmean_ElectronAffinity+wtd_entropy_ElectronAffinity+wtd_range_ElectronAffinity+std_ElectronAffinity+mean_FusionHeat+wtd_range_FusionHeat+std_FusionHeat+mean_ThermalConductivity+wtd_gmean_ThermalConductivity+entropy_ThermalConductivity+wtd_entropy_ThermalConductivity+wtd_range_ThermalConductivity+std_ThermalConductivity+gmean_Valence+range_Valence+wtd_range_Valence+log(gmean_fie) + log(mean_Density) + log(gmean_ElectronAffinity))^2,validation_set)[,-1]


v_interact <- v_interact %>% as.data.frame()
v_interact <- v_interact[which(!names(v_interact) %in% names(validation2.2))]
validation2.2 <- cbind(v_interact,validation2.2)
dim(validation2.2)
## [1] 1544  476

Still the same proceduer preparing test set for regulariztion…

# creates a set of features including interaction terms (test set)
test_interact <- model.matrix(critical_temp~(mean_atomic_mass+std_atomic_mass+wtd_gmean_fie+wtd_range_fie+mean_atomic_radius+wtd_range_atomic_radius+std_atomic_radius+wtd_entropy_Density+wtd_range_Density+wtd_std_Density+wtd_gmean_ElectronAffinity+wtd_entropy_ElectronAffinity+wtd_range_ElectronAffinity+std_ElectronAffinity+mean_FusionHeat+wtd_range_FusionHeat+std_FusionHeat+mean_ThermalConductivity+wtd_gmean_ThermalConductivity+entropy_ThermalConductivity+wtd_entropy_ThermalConductivity+wtd_range_ThermalConductivity+std_ThermalConductivity+gmean_Valence+range_Valence+wtd_range_Valence+log(gmean_fie) + log(mean_Density) + log(gmean_ElectronAffinity))^2,test_set)[,-1]


test_interact <- test_interact %>% as.data.frame()
test_interact <- test_interact[which(!names(test_interact) %in% names(test2.2))]
test2.2 <- cbind(test_interact,test2.2)
dim(test2.2)
## [1] 1544  476

Before we fit data to the Ridge regression, let’s do not forget to convert our dataframes into matices.

# transforms df to matrix as input args of the glmnet()
xmat_train <- training2.2 %>% select(-critical_temp) %>% as.matrix()
ymat_train <- training2.2 %>% select(critical_temp) %>% as.matrix()
xmat_v <- validation2.2 %>% select(-critical_temp) %>% as.matrix()
ymat_v <- validation2.2 %>% select(critical_temp) %>% as.matrix()
xmat_test <- test2.2 %>% select(-critical_temp) %>% as.matrix()
ymat_test <- test2.2 %>% select(critical_temp) %>% as.matrix()


dim(xmat_train)
## [1] 17290   475
dim(xmat_v)
## [1] 1544  475
dim(xmat_test)
## [1] 1544  475

Fit Training Data to L2 Ridge Regression

Now that the data is ready, we ready to fit the data using 10-fold cross validation to optimise labda for L2 Shrinkage Penalty.

# fit a ridge model with cross-validation using the cv.glmnet() function
ridge2.2 <- cv.glmnet(xmat_train, ymat_train, type.measure="mse", 
  alpha=0, family="gaussian")
plot(ridge2.2)

Ridge Assessment on Validation Set

Now we will use the predict() function to apply ridge model to the training set and the validation set using the optimimal lambda value.

The optimal \(\lambda\) can be extracted by lambda.1se, which is \(\lambda\)*, the lambda value resulted in the simplest model (the model with the fewest non-zero parameters). We know that the lambda value was within 1 standard error of the lambda that rsulted in the smallest total cost.

predicts on the validation set

# using the optimised lambda to predict
cat("Optimized lambda =",ridge2.2$lambda.1se)
## Optimized lambda = 2.965807
# Coefficients
coef_ridge2.2 <- coef(ridge2.2)[-1, 1]
coef_ridge2.2 <- coef_ridge2.2[order(abs(coef_ridge2.2), decreasing = TRUE)]

num_coef_ridge2.2 <- length(coef_ridge2.2[coef_ridge2.2 >0])
cat("\nnumber of predictors = ",num_coef_ridge2.2)
## 
## number of predictors =  245
# predicts on training set
pred_train_ridge2.2 <- predict(ridge2.2, s=ridge2.2$lambda.1se,newx=xmat_train)

# MSE
rmse_train_ridge2.2 <- RMSE(pred_train_ridge2.2, training_set$critical_temp)
cat("\nTraining RMSE of Ridge regression on model 2 =",rmse_train_ridge2.2)
## 
## Training RMSE of Ridge regression on model 2 = 16.56204
# Rsquared
rsq_train_ridge2.2 <- cor(pred_train_ridge2.2, training_set$critical_temp)^2
cat("\nTraining adj.rsquared of Ridge regression on model 2 = ",rsq_train_ridge2.2)
## 
## Training adj.rsquared of Ridge regression on model 2 =  0.7687567

predicts on the validation set

# predict on validation set
pred_v_ridge2.2 <- predict(ridge2.2, s=ridge2.2$lambda.1se,newx=xmat_v)

# MSE
rmse_v_ridge2.2 <- RMSE(pred_v_ridge2.2, validation_set$critical_temp)
cat("\nValidation RMSE of Ridge regression on model 2 =",rmse_v_ridge2.2)
## 
## Validation RMSE of Ridge regression on model 2 = 17.5677
# Rsquared
rsq_v_ridge2.2 <- cor(pred_v_ridge2.2, validation_set$critical_temp)^2
cat("\nValidation adj.rsquared of Ridge regression on model 2 = ",rsq_v_ridge2.2)
## 
## Validation adj.rsquared of Ridge regression on model 2 =  0.7232634

Fit Training Data to Lasso Regularisation

Now we move on to fit the data cv.glmnet() to optimise lambda for L1 Shrinkage Penalty.

From the plot blow, we see that as lambda increases, the number of features shrinks, and the mean-squared error increases. When \(\lambda\) approaches zero, we get minimized MSE.

#### alpha = 1, Lasso Regression
################################
# fit a lasso model with cross-validation using the cv.glmnet() function
lasso2.2 <- cv.glmnet(xmat_train, ymat_train, type.measure="mse", 
  alpha=1, family="gaussian")

plot(lasso2.2)

Similarly, we will fit the model to the training set and the validation set using the optimized lambda value.

# min lambda
cat("Optimized lambda =",lasso2.2$lambda.1se)
## Optimized lambda = 0.00624274
# coefficients
coef_lasso2.2 <- coef(lasso2.2)[-1, 1]
coef_lasso2.2 <- coef_lasso2.2[order(abs(coef_lasso2.2), decreasing = TRUE)]
num_coef_lasso2.2 <- length(coef_lasso2.2[coef_lasso2.2 >0])
cat("\nnumber of predictors = ",num_coef_lasso2.2)
## 
## number of predictors =  187
# Predict Tc on Training set
pred_train_lasso2.2 <- predict(lasso2.2, s=lasso2.2$lambda.1se, newx=xmat_train)
# MSE
rmse_train_lasso2.2 <- RMSE(pred_train_lasso2.2, training_set$critical_temp)
cat("\nTraining MSE of Ridge regression on model 2 =",rmse_train_lasso2.2)
## 
## Training MSE of Ridge regression on model 2 = 15.09433
# Rsquared
rsq_train_lasso2.2 <- cor(pred_train_lasso2.2, training_set$critical_temp)^2
cat("\nTraining adj.rsquared of Lasso regression on model 2 = ",rsq_train_lasso2.2)
## 
## Training adj.rsquared of Lasso regression on model 2 =  0.8072066

Lasso Assessment on Validation Set

As we predicted on the validation set using the optimized lambda value, we found that Lasso Regularization gave us better results.

# predict on validation set
pred_v_lasso2.2 <- predict(lasso2.2, s=lasso2.2$lambda.1se, newx=xmat_v)

# MSE
rmse_v_lasso2.2 <- RMSE(pred_v_lasso2.2,validation_set$critical_temp)
cat("\nValidation MSE of Lasso regression on model 2 = ",rmse_v_lasso2.2)
## 
## Validation MSE of Lasso regression on model 2 =  16.58278
# R-squared
rsq_v_lasso2.2 <- cor(pred_v_lasso2.2, validation_set$critical_temp)^2
cat("\nValidation adj.r-squared of Lasso regression on model 2 = ",rsq_v_lasso2.2)
## 
## Validation adj.r-squared of Lasso regression on model 2 =  0.7536771

RMSE Analysis

Comparing the three models, we believe that Lasso did a better job in regulariztion given the better r-squared, lower MSE, and fewer number of coefficients.

The result makes sense because Lasso regression is known for excluding useless variables from equations and making the final equation simpler and easier to interpret (and we assumed that most of the generated terms in our second model are not that important). So it is a better than Ridge regression at reducing the variance in models that contain lots of useless predictors. In contrast, Ridge regression tend to do a little better when most variables are useful.

# creates vectors of stats
lin_reg_trans <- c("num_predictors"=num_features2.1,
             "adj.rsquared_train"=summary(fit2.1)$adj.r.squared,
             "adj.rsquared_validation"=rsq_v2.1,
             "rmse_validation"=rmse_v2.1)

lin_reg_ridge <- c("num_predictors"=num_coef_ridge2.2, 
           "adj.rsquared_train"=rsq_train_ridge2.2,
           #"mse_train"=mse_train_ridge2.2,
           "adj.rsquared_validation"=rsq_v_ridge2.2,
           "rmse_validation"=rmse_v_ridge2.2)

lin_reg_lasso <- c("num_predictors"=num_coef_lasso2.2, 
           "adj.rsquared_train"=rsq_train_lasso2.2,
           #"mse_train"=mse_train_lasso2.2,
           "adj.rsquared_validation"= rsq_v_lasso2.2,
           "rmse_validation"= rmse_v_lasso2.2)

# creates a df by combing above vectors
models.2 <- data.frame(lin_reg_trans,lin_reg_ridge,lin_reg_lasso)
models.2
##                         lin_reg_trans lin_reg_ridge lin_reg_lasso
## num_predictors            477.0000000   245.0000000   187.0000000
## adj.rsquared_train          0.8152777     0.7687567     0.8072066
## adj.rsquared_validation     0.7618830     0.7232634     0.7536771
## rmse_validation.3          16.3187004    17.5677006    16.5827787

Skewness Analysis

Lasso regression achieved 0.77 in rsquared with fewer attributes.

To get a better idea, we took a closer look at the distribution of these important features in the Lasso Model. However, due to the number of features, we only look at those with coefficient > 0.01.

We found that most of the features have weird and wild distributions. Again, let’s try to do log transformation automatically on all of them and see if they look more gaussian

# checks skewness of important features in Lasso model
features <- names(coef_lasso2.2[coef_lasso2.2 >0.01])
features_lasso <- training2.2 %>% select(features)
## Note: Using an external vector in selections is ambiguous.
## ℹ Use `all_of(features)` instead of `features` to silence this message.
## ℹ See <https://tidyselect.r-lib.org/reference/faq-external-vector.html>.
## This message is displayed once per session.
# plot hist
par(mfrow=c(3, 3))
N = ncol(features_lasso)
for (i in 1:N) {
  hist(features_lasso[,i], breaks = 20, main = paste(i,names(features_lasso)[i],sep = ".\n"), border="grey", col="darkgrey")
}

Some of the features became more gaussian in their distribution after log transformation.

  • log(gmean_ElectronAffinity)
  • log(wtd_gmean_ElectronAffinity:gmean_Valence)
# log transformation on features with skewed dist
N = ncol(features_lasso)
colorcode <- rep("grey",N)
colorcode[c(45,53)] <- "deeppink1"

par(mfrow=c(3, 3))
for (i in 1:N) {
  if (min(features_lasso[,i])>0){
    hist(log(features_lasso[,i]), breaks = 20, main = paste(i,paste("log(",names(features_lasso)[i],")",sep = ""),sep = ".\n"), border=colorcode[i], col=colorcode[i],cex.main=0.9)
  }
}

Generalization of Error

Once again, we are going to assess the performance of Model 2.2 (with Lasso Regression). Then, we are going to assume the test errors as an approximation of our generalization error.

# predict on test set
pred_test_lasso2.2 <- predict(lasso2.2, s=lasso2.2$lambda.1se, newx=xmat_test)

# MSE
rmse_test2.2 <- RMSE(pred_test_lasso2.2, test_set$critical_temp)
cat("\nLinear Regression with Regularisation: test MSE = ",rmse_test2.2)
## 
## Linear Regression with Regularisation: test MSE =  16.58278
# R-squared
rsq_test2.2 <- cor(pred_test_lasso2.2, test_set$critical_temp)^2
cat("\nLinear Regression with Regularisation: test adj.rsquared = ",rsq_test2.2)
## 
## Linear Regression with Regularisation: test adj.rsquared =  0.7536771

Visualization of Generalized Error

The visualization on the test results once agian tell the same story. The weak relationship between X values and Y values appeared within the interval of Y < 100.

# Visualizing performance/error on test set
Lasso_regression_test <- ggplot() +
  geom_point(aes(x = test_set$critical_temp, y = pred_test_lasso2.2),
             colour = 'magenta',alpha=0.5,size=3) +
  ggtitle('Lasso Regression') +
  ylab('Prediction') +
  xlab('True Value') + 
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

Lasso_regression_test  

3.1.5 Linear Regression with Feature Crosses

We suspect that the relationship between features and the label was a non-linear one as we saw that the linear regression trainined on the raw features and even the log-transformed ones did poorly on the test set. Here we try to solve a nonlinear problem through feature enginerring to generate feature crosses.

A feature cross is a synthetic feature that encodes nonlinearity in the feature space by multiplying two or more input features together. (The term cross comes from cross product).

Let’s try brute force all the possible combinations of two-way crosses all at once: \(x_3 = x_{1} x_2\) (Although we may suffer from heavy complexity of the model due to a significant increas in the number of features and at at the risk of overfitting, we still wanted to give it a shot!)

Also, due to the skewness we observed in the distribution of critical_temp, our target variable, we are going to take the log of it. Let’s try and see if it helps improve the prediction accuracy.

par(mfrow=c(2, 2))
hist(superconductor$critical_temp, breaks = 25, main = "Tc", border="grey", col="dimgrey")
hist(log(superconductor$critical_temp), breaks = 25, main = "log(Tc)", border="grey", col="dimgrey")

3.1.5.1 Linear Regression on two-way feature crosses

fit data to the linear regression model.

fit3.1 = lm(log(critical_temp)~(.)^2,data = training_set)

This time, we got a very high R-squared of a little over 0.9 using up to 3321 predictors. Although it is a good number to see here, we might still wonder if it is the result of over-fitting given to the complexity.

We will check how good our test data fits the model to decide that this is really a useful model.

num_features3.1 <- dim(summary(fit3.1)$coefficients)[1]-1
cat("Number of features in model 3.1 = ",num_features3.1)
## Number of features in model 3.1 =  3321
cat("\nRsquared = ",summary(fit3.1)$adj.r.squared)
## 
## Rsquared =  0.9066994

Diagnostic plots

We run a residual analysis by plotting the diagnostic plots and try to understand what is going on with the residuals.

  • Residuals vs Fitted : shows the residuals are almost evenly distributed around zero, this suggests that the model doesn’t capture the non-linear relationship
  • Normal Q-Q plot shows that the residuals do not follow the straight line well, indicating non-normal distribution
  • Scale-Location plot tells us the residuals are not randomly spread, violating the assumption of equal variance (homoscedasticity)
  • Residuals vs Leverage shows little evididence of influencial cases
autoplot(fit3.1)

3.1.5.2 Three-way feature crosses on low correlated terms

Now that we have got a good rsquared by generating a complex set of two-way interaction terms: \(x_4 = x_{1} x_2 x_3\), we wondered if three-way get us even better results.

However, the computational complexity of generating three-way interactions (exponential) on all 81 features is likely going to crash our model, so we decided to try to narrow down our base features. Here, we adopted the lowCor subset we created earlier based on the collinearity. So the number of base features went down from 81 to 30. So we expected to see a total of a little more than 4000 indicators( \(\frac {29*29*29}{3!}\) + 29 ) in our three-way-interaction model.

lowCor %>% select(-critical_temp) %>% ncol()
## [1] 29

Fitting around 4089 indicators to the model, we got an rsquared of 0.91 this time on the traning data.

Compared to our two-way-interaction model with 3321 indicators in it, we only improved 0.01 in rsquared value aftering increasing the model complexity by having 768 more terms. This tells us that most of the predictors in are rather useless. Let’s still try predicting Tc on the validation set and assess the performance before we decide which model is better.

fit3.2 = lm(log(critical_temp)~(.)^3,data = lowCor)

# prints stats to console
num_features3.2 <- dim(summary(fit3.2)$coefficients)[1]-1
cat("Number of features in model 3.2 = ",num_features3.2)
## Number of features in model 3.2 =  4089
cat("\nRsquared = ",summary(fit3.2)$adj.r.squared)
## 
## Rsquared =  0.9164246

RMSE of 2-way and 3-way Feature Crosses

Let’s Predict Tc for training set and validation set and calculated the RMSE

## MODEL 3.1
# Predicting Tc for training/validation set
predLog_train3.1 <- predict(fit3.1, newdata = training_set)
pred_train3.1 <- exp(predLog_train3.1)

predLog_v3.1 <- predict(fit3.1, newdata = validation_set)
pred_v3.1 <- exp(predLog_v3.1)

# Calculates RMSE of training pred
rmse_train3.1 <- RMSE(pred_train3.1,training_set$critical_temp)
rmse_v3.1 <- RMSE(pred_v3.1, validation_set$critical_temp)


## MODEL 3.2
# Predicting Tc for training/validation set
predLog_train3.2 <- predict(fit3.2, newdata = training_set)
pred_train3.2 <- exp(predLog_train3.2)

predLog_v3.2 <- predict(fit3.2, newdata = validation_set)
pred_v3.2 <- exp(predLog_v3.2)

# Calculates RMSE of training pred
rmse_train3.2 <- RMSE(pred_train3.2,training_set$critical_temp)
rmse_v3.2 <- RMSE(pred_v3.2, validation_set$critical_temp)

Assessment on Validation Set

From the table we put together below, we found something interesting.

The MSE of predictions of model 3.1 and model 3.2 on the training set was only a little over 12, while it became an incredibly huge number for the validation set. So there’s clearly a sign of overfitting, where we got a low mse on the training set while an extremely bad one on the validation set.

But is it hopeless?

Did the model do a really terrible job predicting Tc in general? Or was there just a few extreme predictions that caused such crazy MSE? Let’s find out by simply plotting the results.

lin_reg_featureCrosses2 <- c("num_predictors"=num_features3.1,
              "adj.rsquared_train"=round(summary(fit3.1)$adj.r.squared,4),
              "rmse_train"=round(rmse_train3.1,4),
              "rmse_validation"="huge")

lin_reg_featureCrosses3 <- c("num_predictors"=num_features3.2,
              "adj.rsquared_train"=round(summary(fit3.2)$adj.r.squared,4),
              "rmse_train"=round(rmse_train3.2,4),
              "rmse_validation"="incredibly huge")

models.featureCrosses <- data.frame(lin_reg_featureCrosses2,lin_reg_featureCrosses3)
models.featureCrosses
##                    lin_reg_featureCrosses2 lin_reg_featureCrosses3
## num_predictors                        3321                    4089
## adj.rsquared_train                  0.9067                  0.9164
## rmse_train.1                        12.571                 12.0242
## rmse_validation                       huge         incredibly huge

Visualization of Error on Training Set

There is a strong relationship going on between the true values and the predicted values. That actually explains why we got such a low mse on the training set.

# Visualizing the predicted values
ggplot() +
  geom_point(aes(x = training_set$critical_temp, y = pred_train3.1),
            colour = 'orangered3',alpha=0.5,size=3) +
  ggtitle('Linear regression model with two-way feature crosses on trainig set') +
  ylab('Prediction') +
  xlab('True Value (Tc)') +
    theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

Visualization of Error on Validation Set

Let’s move on to the test set results.

The plot was bizzare at first glance!

Then as we looked at the scale of y-axis, we soon realized that there seemed to be a few wildly extreme predictions on the validation set.

# Visualizing the predicted values excluding extreme predictions
ggplot() +
  geom_point(aes(x = validation_set$critical_temp, y = pred_v3.1),
            colour = 'orangered3',alpha=0.5,size=3) +
  ggtitle('Linear regression model with two-way feature crosses on validation set') +
  ylab('Prediction') +
  xlab('True Value (Tc)') +
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

As discussed, we suspected there might be some extreme predictions on the validation set.

So we filter and count the number of predicted values that were greater than 134, the maximum value in the validation set.

It turned out that there were 24 extreme values there in the predictions, which account for around 1% of the total predictions. Now let’s filter them out before plotting, otherwise the scale of y-axis would just shift significanly.

high <- max(validation_set$critical_temp)
filtered3.1 <- pred_v3.1[pred_v3.1 < high]
num_extremes3.1 <- length(pred_v3.1) - length(filtered3.1)
cat("Linear regression model with Feature Crosses: Number of extreme predictions on the validation set = ",num_extremes3.1)
## Linear regression model with Feature Crosses: Number of extreme predictions on the validation set =  22
cat("\nLinear regression model with Feature Crosses: Proportion of extreme predictions in the validation set =",num_extremes3.1/nrow(validation_set))
## 
## Linear regression model with Feature Crosses: Proportion of extreme predictions in the validation set = 0.0142487

This time, we got a pretty neat plot. There’s clearly a strong positive corelation between the true values and the predicted ones. As an accurate prediction has an Y=X relationship, we are not too far from that here. We clearly did a better job with this mode than we did with the previous ones.

# Visualizing the predicted values excluding extreme predictions
ggplot() +
  geom_point(aes(x = validation_set$critical_temp[pred_v3.1 < high], y = filtered3.1),
            colour = 'orangered3',alpha=0.5,size=4) +
  ggtitle(paste0('Linear regression model with two-way feature crosses on validation set \n(with ',num_extremes3.1,' extreme predictions removed)')) +
  ylab('Prediction') +
  xlab('True Value (Tc)') +
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

Visualization of Error on Training Set

Now we’ll plot the preformance for model 3.2, which has more than 4000 predictors. Again, there’s a strong linear relationship between X and Y as we fitted the training set.

# Visualizing the predicted values
ggplot() +
  geom_point(aes(x = training_set$critical_temp, y = pred_train3.2),
            colour = 'rosybrown',alpha=0.5,size=3) +
  ggtitle('Linear regression model with three-way feature crosses on training set') +
  ylab('Prediction') +
  xlab('True Value (Tc)') +
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

Visualization of Error on Validation Set

As expected, we got a few of the crazy predictons again. Let’s find out how many of the predictions are overly extreme this time.

# Visualizing the predicted values excluding extreme predictions
ggplot() +
  geom_point(aes(x = validation_set$critical_temp, y = pred_v3.2),
            colour = 'rosybrown',alpha=0.5,size=3) +
  ggtitle('Linear regression model with three-way feature crosses on validation set') +
  ylab('Prediction') +
  xlab('True Value (Tc)') +
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

Turned out there are 43 predicted values > max(test_set.Tc) That was more than double the number we have observed in our model 3.1. Similarly, let’s filter them out and plot again to see the overall performance of model 3.2

filtered3.2 <- pred_v3.2[pred_v3.2 < high]
num_extremes3.2 <- length(pred_v3.2) - length(filtered3.2)
cat("Linear Model with Feature Crosses (3 way): Number of extreme predictions on the validation set = ",num_extremes3.2)
## Linear Model with Feature Crosses (3 way): Number of extreme predictions on the validation set =  29
cat("\nLinear Model with Feature Crosses (3 way): Proportion of extreme predictions in the validation set =",num_extremes3.2/nrow(validation_set))
## 
## Linear Model with Feature Crosses (3 way): Proportion of extreme predictions in the validation set = 0.01878238

After removing those extreme cases, we got a much nicer plot again. The predictions had a strong linear relationship with the true Tc.

# Visualizing the predicted values excluding extreme predictions
ggplot() +
  geom_point(aes(x = validation_set$critical_temp[pred_v3.2 < high], y = filtered3.2),
            colour = 'rosybrown',alpha=0.5,size=4) +
  ggtitle(paste0('Linear Model with Feature Crosses (3 way) Performance on Validation Set \n(with ',num_extremes3.2,' extreme predictions removed)')) +
  ylab('Prediction') +
  xlab('True Value (Tc)') + 
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

Comparing 2-way and 3-way Feature Crosses

Looking at the performance of the two models, we still believe that Model3.1 is a better choice given the number of feature, number of extreme predictions, and overall accuracy in predicting Tc.

Even though clearly both of the models are an example of overfitting, due to time and complexity, we are not going to do feature selection such as stepwise algorithm or regularization as we did before. We decided to leave it as it. Still, we will observe and analyze its performance when we try fitting the test set on it.

# Visualizing Model 3.1 error
p1<- ggplot() +
  geom_point(aes(x = training_set$critical_temp, y = pred_train3.1),colour = 'orangered3',alpha=0.5,size=2) +
  ggtitle('Model 3.1 Performance on Trainig Set') +
  theme(plot.title = element_text(size = 10)) +
  ylab('Prediction') +
  xlab('True Value (Tc)') + theme_light()

# Visualizing Model 3.2 error
p2<- ggplot() +
  geom_point(aes(x = training_set$critical_temp, y = pred_train3.2),colour = 'rosybrown',alpha=0.5,size=2) +
  ggtitle('Model 3.2 Performance on Trainig Set') +
  theme(plot.title = element_text(size = 10)) +
  ylab('Prediction') +
  xlab('True Value (Tc)') + theme_light()

# Visualizing Model 3.1 error after removing wild predictions
p3<- ggplot() +
  geom_point(aes(x = validation_set$critical_temp[pred_v3.1 < high], y = filtered3.1),colour = 'orangered3',alpha=0.5,size=3) +
  ggtitle(paste0('Model 3.1 Performance on Validation Set \n(with ',num_extremes3.1,' extreme predictions removed)')) +
  theme(plot.title = element_text(size = 10)) +
  ylab('Prediction') +
  xlab('True Value (Tc)') + theme_light()

# Visualizing Model 3.2 results after removing wild predictions
p4<- ggplot() +
  geom_point(aes(x = validation_set$critical_temp[pred_v3.2 < high], y = filtered3.2),colour = 'rosybrown',alpha=0.5,size=3) +
  ggtitle(paste0('Model 3.2 Performance on Validation Set \n(with ',num_extremes3.2,' extreme predictions removed)')) +
  theme(plot.title = element_text(size = 10)) +
  ylab('Prediction') +
  xlab('True Value (Tc)') + theme_light()

grid.arrange(p1, p2, p3,p4,nrow=2)

Generalization of Error

Now that we have chosen our third model,we are going to assess the performance of our model on the test set as usual. We already expected to see a significantly large MSE due to extreme predictions we observed in the training set and the validation set results.

# Predicting Tc on test set
predLog_test3.1 <- predict(fit3.1, newdata = test_set)
pred_test3.1 <- exp(predLog_test3.1)

# Calculates RMSE
rmse_test3.1 <- RMSE(pred_test3.1, test_set$critical_temp)
cat("\nMODEL 3.1: RMSE for the test predictions =", rmse_test3.1)
## 
## MODEL 3.1: RMSE for the test predictions = 758819.2

Visualization of Genralized Error

Judging by the MSE, it is obvious that there are extreme predictions for sure. Turns out its less than 1%, not too bad. Let’s filter them out before plotting.

#high <- max(validation_set$critical_temp)
filtered <- pred_test3.1[pred_test3.1 < high]
num_extremes <- length(pred_test3.1) - length(filtered)

# rsquared and mse after removing extremes 
rsq_test3.1 <- cor(filtered, test_set$critical_temp[pred_test3.1 < high])^2
adj.rmse_test3.1 <- RMSE(filtered,test_set$critical_temp[pred_test3.1 < high])

cat("MODEL3.1: Number of extreme predictions on the test set = ",num_extremes)
## MODEL3.1: Number of extreme predictions on the test set =  22
cat("\nMODEL3.1: Proportion of extreme predictions in the test set =",num_extremes/nrow(test_set))
## 
## MODEL3.1: Proportion of extreme predictions in the test set = 0.0142487
cat("\n(After removing extreme observations) test RMSE =",adj.rmse_test3.1)
## 
## (After removing extreme observations) test RMSE = 13.98703
#cat("\n(After removing extreme observations) adj.rsquared =",rsq_test3.1)

We are going to fit the test set to our extremely complex third model and visualize the prediction. From the plot below, we can see a strong linear relationship between true values and the predicted values. Although we got around 1% extreme predictions on the test set, we still did a fairly good job on predicting Tc using our third model.

# Visualizing generalization error
featureCrosses_test <- ggplot() +
  geom_point(aes(x = test_set$critical_temp[pred_test3.1 < high], y = filtered),
             colour = 'orangered3',alpha=0.5,size=3) +
  ggtitle(paste0('Feature Crosses (with ',num_extremes,' extreme predictions removed)')) +
  ylab('Prediction') +
  xlab('True Value') +
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

featureCrosses_test 

3.2 Random Forest

Random Forest algorithm is an ensemble learning method for classification as well as regression that leverages the power of multiple decision trees for making decisions. Not only is this algorithm better than linear regression at capturing non-linear relationships, by introducing a technique called bagging to the traditional Decision Tree Algorithm, it also improves the problem of overfitting to the training data.

Bagging (Bootstrap aggregating) here is a method designed to reduces variance and helps to avoid overfitting. By sampling from a training set \(D\) uniformly(randomly) and with replacement, we get \(n\) different bootstrap samples for building \(m\) decision trees. We then combine the results of the \(m\) tress by averaging the output (for regression) or voting (for classification).

random_forest_model <- train(
critical_temp ~., data = training_set,
trControl = trainControl( method = "cv", number = 10, search = "random"), tuneLength = 5,
method = "ranger",
importance = 'impurity',
preProc = c("range")
)
## Growing trees.. Progress: 47%. Estimated remaining time: 34 seconds.
## Growing trees.. Progress: 96%. Estimated remaining time: 2 seconds.
## Growing trees.. Progress: 44%. Estimated remaining time: 39 seconds.
## Growing trees.. Progress: 89%. Estimated remaining time: 7 seconds.
## Growing trees.. Progress: 41%. Estimated remaining time: 44 seconds.
## Growing trees.. Progress: 80%. Estimated remaining time: 15 seconds.
## Growing trees.. Progress: 66%. Estimated remaining time: 15 seconds.
## Growing trees.. Progress: 51%. Estimated remaining time: 29 seconds.
## Growing trees.. Progress: 45%. Estimated remaining time: 37 seconds.
## Growing trees.. Progress: 90%. Estimated remaining time: 7 seconds.
## Growing trees.. Progress: 42%. Estimated remaining time: 43 seconds.
## Growing trees.. Progress: 84%. Estimated remaining time: 11 seconds.
## Growing trees.. Progress: 68%. Estimated remaining time: 14 seconds.
## Growing trees.. Progress: 50%. Estimated remaining time: 31 seconds.
## Growing trees.. Progress: 47%. Estimated remaining time: 35 seconds.
## Growing trees.. Progress: 91%. Estimated remaining time: 6 seconds.
## Growing trees.. Progress: 41%. Estimated remaining time: 44 seconds.
## Growing trees.. Progress: 82%. Estimated remaining time: 13 seconds.
## Growing trees.. Progress: 69%. Estimated remaining time: 13 seconds.
## Growing trees.. Progress: 46%. Estimated remaining time: 35 seconds.
## Growing trees.. Progress: 96%. Estimated remaining time: 2 seconds.
## Growing trees.. Progress: 44%. Estimated remaining time: 38 seconds.
## Growing trees.. Progress: 91%. Estimated remaining time: 5 seconds.
## Growing trees.. Progress: 31%. Estimated remaining time: 1 minute, 7 seconds.
## Growing trees.. Progress: 52%. Estimated remaining time: 58 seconds.
## Growing trees.. Progress: 87%. Estimated remaining time: 13 seconds.
## Growing trees.. Progress: 61%. Estimated remaining time: 19 seconds.
## Growing trees.. Progress: 51%. Estimated remaining time: 29 seconds.
## Growing trees.. Progress: 43%. Estimated remaining time: 41 seconds.
## Growing trees.. Progress: 87%. Estimated remaining time: 8 seconds.
## Growing trees.. Progress: 41%. Estimated remaining time: 44 seconds.
## Growing trees.. Progress: 83%. Estimated remaining time: 13 seconds.
## Growing trees.. Progress: 65%. Estimated remaining time: 16 seconds.
## Growing trees.. Progress: 48%. Estimated remaining time: 33 seconds.
## Growing trees.. Progress: 99%. Estimated remaining time: 0 seconds.
## Growing trees.. Progress: 45%. Estimated remaining time: 37 seconds.
## Growing trees.. Progress: 90%. Estimated remaining time: 6 seconds.
## Growing trees.. Progress: 40%. Estimated remaining time: 46 seconds.
## Growing trees.. Progress: 67%. Estimated remaining time: 30 seconds.
## Growing trees.. Progress: 62%. Estimated remaining time: 19 seconds.
## Growing trees.. Progress: 49%. Estimated remaining time: 32 seconds.
## Growing trees.. Progress: 99%. Estimated remaining time: 0 seconds.
## Growing trees.. Progress: 43%. Estimated remaining time: 41 seconds.
## Growing trees.. Progress: 85%. Estimated remaining time: 11 seconds.
## Growing trees.. Progress: 34%. Estimated remaining time: 1 minute, 0 seconds.
## Growing trees.. Progress: 70%. Estimated remaining time: 26 seconds.
## Growing trees.. Progress: 61%. Estimated remaining time: 20 seconds.
## Growing trees.. Progress: 47%. Estimated remaining time: 34 seconds.
## Growing trees.. Progress: 96%. Estimated remaining time: 2 seconds.
## Growing trees.. Progress: 44%. Estimated remaining time: 39 seconds.
## Growing trees.. Progress: 89%. Estimated remaining time: 7 seconds.
## Growing trees.. Progress: 39%. Estimated remaining time: 48 seconds.
## Growing trees.. Progress: 78%. Estimated remaining time: 17 seconds.
## Growing trees.. Progress: 60%. Estimated remaining time: 21 seconds.
## Growing trees.. Progress: 49%. Estimated remaining time: 31 seconds.
## Growing trees.. Progress: 98%. Estimated remaining time: 1 seconds.
## Growing trees.. Progress: 46%. Estimated remaining time: 36 seconds.
## Growing trees.. Progress: 91%. Estimated remaining time: 5 seconds.
## Growing trees.. Progress: 41%. Estimated remaining time: 44 seconds.
## Growing trees.. Progress: 82%. Estimated remaining time: 13 seconds.
## Growing trees.. Progress: 60%. Estimated remaining time: 20 seconds.
## Growing trees.. Progress: 48%. Estimated remaining time: 33 seconds.
## Growing trees.. Progress: 98%. Estimated remaining time: 1 seconds.
## Growing trees.. Progress: 47%. Estimated remaining time: 35 seconds.
## Growing trees.. Progress: 85%. Estimated remaining time: 11 seconds.
## Growing trees.. Progress: 38%. Estimated remaining time: 51 seconds.
## Growing trees.. Progress: 78%. Estimated remaining time: 17 seconds.
## Growing trees.. Progress: 65%. Estimated remaining time: 16 seconds.
## Growing trees.. Progress: 59%. Estimated remaining time: 21 seconds.
# random forest training results
random_forest_model$results
##   min.node.size mtry  splitrule     RMSE  Rsquared      MAE    RMSESD
## 4            13   10 extratrees 9.677616 0.9213217 5.682707 0.4860627
## 1             2   16    maxstat 9.446636 0.9247586 5.388818 0.4859528
## 2             6   24    maxstat 9.552107 0.9231087 5.467311 0.4914655
## 3             8   28    maxstat 9.611306 0.9222023 5.514149 0.5018636
## 5            19   35    maxstat 9.990829 0.9163405 5.853876 0.5169983
##    RsquaredSD     MAESD
## 4 0.007411564 0.1870933
## 1 0.007221282 0.1571687
## 2 0.007378714 0.1528034
## 3 0.007588542 0.1506292
## 5 0.008185058 0.1570023
# random forest with optimised tuning perameters
random_forest_model$bestTune
##   mtry splitrule min.node.size
## 1   16   maxstat             2
random_forest_model$finalModel
## Ranger result
## 
## Call:
##  ranger::ranger(dependent.variable.name = ".outcome", data = x,      mtry = min(param$mtry, ncol(x)), min.node.size = param$min.node.size,      splitrule = as.character(param$splitrule), write.forest = TRUE,      probability = classProbs, ...) 
## 
## Type:                             Regression 
## Number of trees:                  500 
## Sample size:                      17290 
## Number of independent variables:  81 
## Mtry:                             16 
## Target node size:                 2 
## Variable importance mode:         impurity 
## Splitrule:                        maxstat 
## OOB prediction error (MSE):       86.47003 
## R squared (OOB):                  0.9268189
# List of features with their importance scores
rf_features <- varImp(random_forest_model)
print(rf_features$importance)
##                                     Overall
## number_of_elements                8.6845292
## mean_atomic_mass                 15.8868933
## wtd_mean_atomic_mass             59.1544978
## gmean_atomic_mass                13.3642900
## wtd_gmean_atomic_mass            56.5833917
## entropy_atomic_mass               1.1004045
## wtd_entropy_atomic_mass          63.1820627
## range_atomic_mass                28.3385286
## wtd_range_atomic_mass            70.8850100
## std_atomic_mass                  13.6316424
## wtd_std_atomic_mass              64.6598559
## mean_fie                         14.8357405
## wtd_mean_fie                     58.6167339
## gmean_fie                        17.1887229
## wtd_gmean_fie                    57.6471647
## entropy_fie                       0.2422649
## wtd_entropy_fie                  55.1561265
## range_fie                        19.8088822
## wtd_range_fie                    70.1847525
## std_fie                           3.8029914
## wtd_std_fie                      67.6328522
## mean_atomic_radius               28.8765014
## wtd_mean_atomic_radius           66.6681244
## gmean_atomic_radius              13.3784045
## wtd_gmean_atomic_radius          60.6103713
## entropy_atomic_radius             3.1937353
## wtd_entropy_atomic_radius        62.4846252
## range_atomic_radius              26.7373524
## wtd_range_atomic_radius          72.3394854
## std_atomic_radius                12.2705662
## wtd_std_atomic_radius            70.8406127
## mean_Density                      8.8726992
## wtd_mean_Density                 57.7426850
## gmean_Density                     4.9821414
## wtd_gmean_Density                54.2331090
## entropy_Density                   4.0194418
## wtd_entropy_Density              71.2375102
## range_Density                    28.7959942
## wtd_range_Density                71.9234605
## std_Density                      13.4396785
## wtd_std_Density                  65.8824058
## mean_ElectronAffinity            23.8004288
## wtd_mean_ElectronAffinity        71.5723351
## gmean_ElectronAffinity           24.2157853
## wtd_gmean_ElectronAffinity       78.7209820
## entropy_ElectronAffinity         11.4713598
## wtd_entropy_ElectronAffinity     69.2795560
## range_ElectronAffinity           31.3870785
## wtd_range_ElectronAffinity       79.0258032
## std_ElectronAffinity             15.1933557
## wtd_std_ElectronAffinity         75.1397141
## mean_FusionHeat                   8.2237335
## wtd_mean_FusionHeat              63.0281014
## gmean_FusionHeat                  6.4177293
## wtd_gmean_FusionHeat             59.3875950
## entropy_FusionHeat                2.3745707
## wtd_entropy_FusionHeat           61.1154316
## range_FusionHeat                 29.8008345
## wtd_range_FusionHeat             69.6382108
## std_FusionHeat                   14.2143830
## wtd_std_FusionHeat               73.0560909
## mean_ThermalConductivity         17.3026121
## wtd_mean_ThermalConductivity     76.4047757
## gmean_ThermalConductivity        11.1037361
## wtd_gmean_ThermalConductivity    71.1468012
## entropy_ThermalConductivity      15.9099027
## wtd_entropy_ThermalConductivity  79.0912514
## range_ThermalConductivity         0.0000000
## wtd_range_ThermalConductivity    81.1859772
## std_ThermalConductivity          10.1293290
## wtd_std_ThermalConductivity      75.2437346
## mean_Valence                     40.1334009
## wtd_mean_Valence                100.0000000
## gmean_Valence                    27.4427536
## wtd_gmean_Valence                90.7907407
## entropy_Valence                  24.4838578
## wtd_entropy_Valence              75.3036692
## range_Valence                    17.6967426
## wtd_range_Valence                89.4173170
## std_Valence                      42.9398988
## wtd_std_Valence                  91.0774620

Generalization of Error

# Predicting Tc for training/test
pred_test_rf = predict(random_forest_model, newdata = test_set)
rmse_test_rf <- RMSE(pred_test_rf,test_set$critical_temp)
# Calculates RMSE of training pred
cat("\nRandom Forest Model: RMSE for the test predictions =", rmse_test_rf)
## 
## Random Forest Model: RMSE for the test predictions = 10.21365

Visualization of Gernerailzed Error

# Visualizing the fit
rf_test <- ggplot() +
  geom_point(aes(x = test_set$critical_temp, y = pred_test_rf),
            colour = 'gold',alpha=0.5,size=3) +
  ggtitle('Random Forest') +
  ylab('Prediction') +
  xlab('True Value (Tc)') + 
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

rf_test

3.3 Gradient Boosting

In addition to Random Forest, we will try another ensemble learning method called Gradient Boosting. Similar to Random Firest algorithm, it builds a set of decision tress with bootstrapping and then combine the results to make a prediction.

However, the way they build those trees are different. Random Forests build each tree independently, thus it doesnt matter in what order you build the tress. On the other hand, Gradient Boosting algorithm builds one tree at a time, introducing a new tree to improve the mistakes (residuals) made by the previous one by trying to minimise the loss using gradient descent algorithm. Similar to other boosting methods, gradient boosting combines weak “learners” into a single strong learner in such iterative fashion.

# Gradient Boosting Machine
gb_model <- train(
  critical_temp ~ ., data = training_set,
  method = "gbm",
  tuneLength = 3,
  preProc = c("range"),
  trControl = trainControl( method = "cv", number = 10, search = "random", verbose = FALSE)
) 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      820.2913             nan     0.2306  360.3121
##      2      598.6821             nan     0.2306  219.2438
##      3      458.8542             nan     0.2306  138.1983
##      4      373.7684             nan     0.2306   83.2345
##      5      315.4238             nan     0.2306   54.8164
##      6      277.8400             nan     0.2306   36.3623
##      7      251.1454             nan     0.2306   25.3507
##      8      232.6973             nan     0.2306   18.0249
##      9      219.7588             nan     0.2306   12.2491
##     10      208.3323             nan     0.2306   11.0855
##     20      163.8643             nan     0.2306    1.0526
##     40      131.4026             nan     0.2306    0.5118
##     60      115.1072             nan     0.2306    0.3414
##     80      105.9440             nan     0.2306    0.0299
##    100       97.7523             nan     0.2306   -0.0345
##    120       89.6829             nan     0.2306    0.0948
##    140       83.3906             nan     0.2306    0.0341
##    160       78.3659             nan     0.2306   -0.2009
##    180       74.8566             nan     0.2306   -0.0881
##    200       71.1983             nan     0.2306   -0.2661
##    220       67.7012             nan     0.2306   -0.1934
##    240       64.7260             nan     0.2306   -0.2320
##    260       62.2020             nan     0.2306   -0.1965
##    280       60.0036             nan     0.2306   -0.0816
##    300       58.2190             nan     0.2306   -0.1936
##    320       55.9247             nan     0.2306   -0.0902
##    340       54.3282             nan     0.2306   -0.1183
##    360       52.7059             nan     0.2306   -0.1692
##    380       51.3612             nan     0.2306   -0.1564
##    400       50.0680             nan     0.2306   -0.2155
##    420       48.8575             nan     0.2306   -0.1558
##    440       47.6979             nan     0.2306   -0.0837
##    460       46.8128             nan     0.2306   -0.1121
##    480       45.9042             nan     0.2306   -0.1624
##    500       44.9538             nan     0.2306   -0.1455
##    520       44.0979             nan     0.2306   -0.1494
##    540       43.3918             nan     0.2306   -0.1428
##    560       42.7064             nan     0.2306   -0.1303
##    580       41.9847             nan     0.2306   -0.0993
##    600       41.1431             nan     0.2306   -0.1838
##    620       40.5524             nan     0.2306   -0.1410
##    640       39.9493             nan     0.2306   -0.0605
##    660       39.2882             nan     0.2306   -0.0877
##    680       38.7405             nan     0.2306   -0.1226
##    700       38.2163             nan     0.2306   -0.1081
##    720       37.6423             nan     0.2306   -0.1124
##    740       37.1198             nan     0.2306   -0.1088
##    760       36.7274             nan     0.2306   -0.1317
##    780       36.3968             nan     0.2306   -0.0921
##    800       36.0118             nan     0.2306   -0.1603
##    820       35.5982             nan     0.2306   -0.1107
##    840       35.2645             nan     0.2306   -0.0910
##    860       34.8935             nan     0.2306   -0.1514
##    880       34.5733             nan     0.2306   -0.1917
##    900       34.3070             nan     0.2306   -0.1004
##    920       33.9854             nan     0.2306   -0.1446
##    940       33.6604             nan     0.2306   -0.1147
##    960       33.4058             nan     0.2306   -0.1391
##    980       33.0602             nan     0.2306   -0.0867
##   1000       32.8290             nan     0.2306   -0.1165
##   1020       32.5714             nan     0.2306   -0.1629
##   1040       32.3317             nan     0.2306   -0.1460
##   1060       32.1684             nan     0.2306   -0.0894
##   1080       31.9288             nan     0.2306   -0.0630
##   1100       31.7255             nan     0.2306   -0.1002
##   1120       31.5634             nan     0.2306   -0.1715
##   1140       31.3720             nan     0.2306   -0.1794
##   1160       31.0873             nan     0.2306   -0.0874
##   1180       30.9373             nan     0.2306   -0.1035
##   1200       30.7372             nan     0.2306   -0.2175
##   1220       30.4989             nan     0.2306   -0.0935
##   1240       30.3589             nan     0.2306   -0.0984
##   1260       30.1654             nan     0.2306   -0.0604
##   1280       29.9929             nan     0.2306   -0.1216
##   1300       29.7950             nan     0.2306   -0.1415
##   1320       29.6314             nan     0.2306   -0.0903
##   1340       29.4563             nan     0.2306   -0.1152
##   1360       29.2934             nan     0.2306   -0.1568
##   1380       29.1701             nan     0.2306   -0.1029
##   1400       29.0160             nan     0.2306   -0.1165
##   1420       28.8555             nan     0.2306   -0.1610
##   1440       28.7316             nan     0.2306   -0.1167
##   1460       28.6386             nan     0.2306   -0.0900
##   1480       28.5134             nan     0.2306   -0.1345
##   1500       28.4165             nan     0.2306   -0.2115
##   1520       28.2953             nan     0.2306   -0.2164
##   1540       28.1938             nan     0.2306   -0.1576
##   1560       28.0855             nan     0.2306   -0.2179
##   1580       27.9710             nan     0.2306   -0.1375
##   1600       27.8689             nan     0.2306   -0.1397
##   1620       27.7130             nan     0.2306   -0.1156
##   1640       27.6342             nan     0.2306   -0.1429
##   1660       27.5094             nan     0.2306   -0.1010
##   1680       27.5106             nan     0.2306   -0.1106
##   1700       27.3978             nan     0.2306   -0.1043
##   1720       27.2967             nan     0.2306   -0.0367
##   1740       27.1649             nan     0.2306   -0.1097
##   1760       27.0857             nan     0.2306   -0.1493
##   1780       26.9950             nan     0.2306   -0.1102
##   1800       26.9209             nan     0.2306   -0.0597
##   1820       26.8076             nan     0.2306   -0.1342
##   1840       26.7148             nan     0.2306   -0.1491
##   1860       26.6261             nan     0.2306   -0.1361
##   1880       26.5313             nan     0.2306   -0.0910
##   1900       26.4739             nan     0.2306   -0.1533
##   1920       26.3820             nan     0.2306   -0.1307
##   1940       26.2836             nan     0.2306   -0.0640
##   1960       26.2352             nan     0.2306   -0.1835
##   1980       26.1312             nan     0.2306   -0.1352
##   2000       26.0805             nan     0.2306   -0.1031
##   2020       26.0557             nan     0.2306   -0.1113
##   2040       25.9696             nan     0.2306   -0.0860
##   2060       25.8802             nan     0.2306   -0.1068
##   2080       25.8650             nan     0.2306   -0.0787
##   2100       25.7584             nan     0.2306   -0.1191
##   2120       25.7463             nan     0.2306   -0.0644
##   2140       25.6889             nan     0.2306   -0.1365
##   2160       25.6601             nan     0.2306   -0.1187
##   2180       25.6439             nan     0.2306   -0.1453
##   2200       25.5546             nan     0.2306   -0.0447
##   2220       25.5277             nan     0.2306   -0.1155
##   2240       25.4004             nan     0.2306   -0.1281
##   2260       25.3826             nan     0.2306   -0.1152
##   2280       25.3321             nan     0.2306   -0.1384
##   2300       25.2527             nan     0.2306   -0.1650
##   2320       25.2276             nan     0.2306   -0.1108
##   2340       25.2350             nan     0.2306   -0.2275
##   2360       25.1238             nan     0.2306   -0.0568
##   2380       25.0739             nan     0.2306   -0.1109
##   2400       25.1065             nan     0.2306   -0.1050
##   2420       25.0529             nan     0.2306   -0.1090
##   2440       25.0301             nan     0.2306   -0.1336
##   2460       24.9988             nan     0.2306   -0.0874
##   2480       24.9461             nan     0.2306   -0.1137
##   2500       24.8900             nan     0.2306   -0.1126
##   2520       24.8626             nan     0.2306   -0.2967
##   2540       24.8382             nan     0.2306   -0.1438
##   2560       24.7570             nan     0.2306   -0.1385
##   2580       24.7368             nan     0.2306   -0.1311
##   2600       24.6795             nan     0.2306   -0.1092
##   2620       24.6338             nan     0.2306   -0.1396
##   2640       24.5671             nan     0.2306   -0.1368
##   2660       24.5834             nan     0.2306   -0.3166
##   2680       24.5194             nan     0.2306   -0.1249
##   2700       24.5075             nan     0.2306   -0.1874
##   2720       24.4778             nan     0.2306   -0.1905
##   2740       24.4080             nan     0.2306   -0.1227
##   2760       24.3755             nan     0.2306   -0.1121
##   2780       24.3471             nan     0.2306   -0.1290
##   2800       24.2629             nan     0.2306   -0.1238
##   2820       24.2342             nan     0.2306   -0.1832
##   2840       24.2029             nan     0.2306   -0.1271
##   2860       24.1922             nan     0.2306   -0.0790
##   2880       24.2217             nan     0.2306   -0.1198
##   2900       24.1293             nan     0.2306   -0.1128
##   2920       24.0930             nan     0.2306   -0.1847
##   2940       24.0430             nan     0.2306   -0.0950
##   2960       24.0120             nan     0.2306   -0.0868
##   2980       23.9303             nan     0.2306   -0.0435
##   3000       23.9434             nan     0.2306   -0.0825
##   3020       23.9519             nan     0.2306   -0.0914
##   3040       23.9029             nan     0.2306   -0.0851
##   3060       23.9317             nan     0.2306   -0.1806
##   3080       23.8765             nan     0.2306   -0.1483
##   3100       23.8262             nan     0.2306   -0.0634
##   3120       23.8491             nan     0.2306   -0.1528
##   3140       23.7744             nan     0.2306   -0.1464
##   3160       23.7189             nan     0.2306   -0.1142
##   3180       23.7124             nan     0.2306   -0.0769
##   3200       23.6925             nan     0.2306   -0.1048
##   3220       23.6352             nan     0.2306   -0.1246
##   3240       23.6419             nan     0.2306   -0.1481
##   3260       23.6003             nan     0.2306   -0.0593
##   3280       23.6261             nan     0.2306   -0.1442
##   3300       23.5481             nan     0.2306   -0.1133
##   3320       23.5229             nan     0.2306   -0.0772
##   3340       23.4981             nan     0.2306   -0.1451
##   3360       23.4716             nan     0.2306   -0.0901
##   3380       23.4419             nan     0.2306   -0.0773
##   3400       23.4127             nan     0.2306   -0.0964
##   3420       23.3991             nan     0.2306   -0.1600
##   3440       23.4046             nan     0.2306   -0.1901
##   3460       23.3581             nan     0.2306   -0.0651
##   3480       23.3325             nan     0.2306   -0.2195
##   3500       23.3037             nan     0.2306   -0.1776
##   3520       23.2473             nan     0.2306   -0.1053
##   3540       23.2146             nan     0.2306   -0.1009
##   3560       23.2291             nan     0.2306   -0.1352
##   3580       23.2069             nan     0.2306   -0.1686
##   3600       23.1637             nan     0.2306   -0.0974
##   3620       23.1529             nan     0.2306   -0.0468
##   3640       23.1079             nan     0.2306   -0.1448
##   3660       23.1358             nan     0.2306   -0.0604
##   3680       23.1074             nan     0.2306   -0.1201
##   3700       23.0796             nan     0.2306   -0.1680
##   3720       23.0987             nan     0.2306   -0.2064
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      671.7691             nan     0.3896  501.6199
##      2      474.4354             nan     0.3896  195.3405
##      3      377.6369             nan     0.3896   96.6095
##      4      328.6915             nan     0.3896   47.3673
##      5      295.8788             nan     0.3896   29.8144
##      6      278.0775             nan     0.3896   17.7912
##      7      264.9746             nan     0.3896   12.1795
##      8      255.9029             nan     0.3896    8.5932
##      9      248.8889             nan     0.3896    5.7788
##     10      240.9638             nan     0.3896    7.3431
##     20      206.6818             nan     0.3896    1.9613
##     40      172.4850             nan     0.3896    1.0346
##     60      154.3235             nan     0.3896    0.3021
##     80      141.8703             nan     0.3896   -0.2541
##    100      133.5588             nan     0.3896   -0.2647
##    120      126.5693             nan     0.3896   -0.0190
##    140      121.7164             nan     0.3896   -0.1568
##    160      115.3749             nan     0.3896   -0.0796
##    180      111.5530             nan     0.3896   -0.1101
##    200      108.4061             nan     0.3896   -0.0267
##    220      105.0297             nan     0.3896   -0.2647
##    240      102.0045             nan     0.3896   -0.0548
##    260       99.2862             nan     0.3896   -0.2295
##    280       96.7103             nan     0.3896   -0.0604
##    300       94.3827             nan     0.3896   -0.4061
##    320       91.6240             nan     0.3896   -0.0495
##    340       89.7554             nan     0.3896   -0.0432
##    360       87.5360             nan     0.3896   -0.1834
##    380       85.3331             nan     0.3896   -0.1846
##    400       83.7717             nan     0.3896   -0.1872
##    420       82.1428             nan     0.3896   -0.1317
##    440       80.5306             nan     0.3896   -0.1857
##    460       79.2666             nan     0.3896   -0.0648
##    480       78.0451             nan     0.3896   -0.1527
##    500       77.0711             nan     0.3896   -0.1370
##    520       75.3480             nan     0.3896   -0.0973
##    540       74.0730             nan     0.3896   -0.2305
##    560       73.0420             nan     0.3896   -0.0689
##    580       72.2410             nan     0.3896   -0.2052
##    600       71.1474             nan     0.3896   -0.1227
##    620       69.9681             nan     0.3896   -0.0758
##    640       69.1475             nan     0.3896   -0.1941
##    660       67.9939             nan     0.3896   -0.1398
##    680       67.2168             nan     0.3896   -0.1334
##    700       66.3240             nan     0.3896   -0.1215
##    720       65.6605             nan     0.3896   -0.0365
##    740       64.8266             nan     0.3896   -0.0781
##    760       63.9765             nan     0.3896   -0.1381
##    780       63.2592             nan     0.3896   -0.1404
##    800       62.5115             nan     0.3896   -0.1707
##    820       62.0700             nan     0.3896   -0.1791
##    840       61.4731             nan     0.3896   -0.0833
##    860       60.8101             nan     0.3896   -0.2099
##    880       60.3130             nan     0.3896   -0.5128
##    900       59.6949             nan     0.3896   -0.1624
##    920       59.0816             nan     0.3896   -0.0988
##    940       58.6371             nan     0.3896   -0.0170
##    960       58.1970             nan     0.3896   -0.0589
##    980       57.3731             nan     0.3896   -0.0842
##   1000       56.8640             nan     0.3896   -0.0825
##   1020       56.3813             nan     0.3896   -0.0871
##   1040       55.8749             nan     0.3896   -0.1240
##   1060       55.3825             nan     0.3896   -0.0421
##   1080       54.7306             nan     0.3896   -0.1590
##   1100       54.3654             nan     0.3896   -0.0427
##   1120       53.8275             nan     0.3896   -0.0261
##   1140       53.3908             nan     0.3896   -0.1098
##   1160       53.0258             nan     0.3896   -0.2421
##   1180       52.6596             nan     0.3896   -0.1003
##   1200       52.2419             nan     0.3896   -0.0690
##   1220       51.8385             nan     0.3896   -0.1322
##   1240       51.3682             nan     0.3896   -0.0773
##   1260       51.0852             nan     0.3896   -0.1177
##   1280       50.6896             nan     0.3896   -0.1070
##   1300       50.3952             nan     0.3896   -0.1382
##   1320       50.0191             nan     0.3896   -0.0859
##   1340       49.7160             nan     0.3896   -0.1043
##   1360       49.5169             nan     0.3896   -0.0988
##   1380       49.2134             nan     0.3896   -0.0785
##   1400       49.0756             nan     0.3896   -0.0580
##   1420       48.8089             nan     0.3896   -0.1056
##   1440       48.4608             nan     0.3896   -0.1440
##   1460       48.0499             nan     0.3896   -0.0908
##   1480       47.6970             nan     0.3896   -0.0576
##   1500       47.5189             nan     0.3896   -0.0790
##   1520       47.3030             nan     0.3896   -0.1736
##   1540       47.0231             nan     0.3896   -0.1310
##   1560       46.7769             nan     0.3896   -0.0763
##   1580       46.6143             nan     0.3896   -0.1641
##   1600       46.3225             nan     0.3896   -0.1014
##   1620       46.0970             nan     0.3896   -0.0779
##   1640       45.9173             nan     0.3896   -0.0890
##   1660       45.6981             nan     0.3896   -0.1062
##   1680       45.5261             nan     0.3896   -0.0778
##   1700       45.2996             nan     0.3896   -0.0604
##   1720       45.1532             nan     0.3896   -0.0993
##   1740       44.9579             nan     0.3896   -0.1452
##   1760       44.7753             nan     0.3896   -0.0942
##   1780       44.5757             nan     0.3896   -0.1614
##   1800       44.3033             nan     0.3896   -0.1551
##   1820       44.0760             nan     0.3896   -0.1071
##   1840       43.9670             nan     0.3896   -0.1256
##   1860       43.8818             nan     0.3896   -0.1829
##   1880       43.7356             nan     0.3896   -0.1085
##   1900       43.4918             nan     0.3896   -0.0669
##   1920       43.3152             nan     0.3896   -0.0243
##   1940       43.1474             nan     0.3896   -0.1047
##   1960       43.0721             nan     0.3896   -0.0430
##   1980       42.8295             nan     0.3896   -0.0893
##   2000       42.5775             nan     0.3896   -0.0999
##   2020       42.4204             nan     0.3896   -0.0681
##   2040       42.2921             nan     0.3896   -0.0633
##   2060       42.1689             nan     0.3896   -0.1324
##   2080       41.9684             nan     0.3896   -0.0552
##   2100       41.8009             nan     0.3896   -0.0821
##   2120       41.6404             nan     0.3896   -0.0882
##   2140       41.4944             nan     0.3896   -0.0880
##   2160       41.4587             nan     0.3896   -0.1410
##   2180       41.2788             nan     0.3896   -0.0872
##   2200       41.1513             nan     0.3896   -0.1958
##   2220       41.0049             nan     0.3896   -0.1596
##   2240       40.7971             nan     0.3896   -0.0520
##   2260       40.5702             nan     0.3896   -0.0399
##   2280       40.4627             nan     0.3896   -0.1076
##   2300       40.3633             nan     0.3896   -0.0590
##   2320       40.2796             nan     0.3896   -0.1095
##   2340       40.1879             nan     0.3896   -0.1353
##   2360       40.0313             nan     0.3896   -0.0627
##   2380       39.9332             nan     0.3896   -0.0622
##   2400       39.8443             nan     0.3896   -0.1181
##   2420       39.7230             nan     0.3896   -0.0285
##   2440       39.6195             nan     0.3896   -0.0676
##   2460       39.4445             nan     0.3896   -0.0467
##   2480       39.3484             nan     0.3896   -0.1022
##   2500       39.1918             nan     0.3896   -0.0957
##   2520       39.1115             nan     0.3896   -0.1106
##   2540       39.0168             nan     0.3896   -0.1716
##   2560       38.8884             nan     0.3896   -0.0670
##   2580       38.7715             nan     0.3896   -0.0930
##   2600       38.7386             nan     0.3896   -0.0880
##   2620       38.6067             nan     0.3896   -0.0490
##   2640       38.5340             nan     0.3896   -0.0877
##   2660       38.3115             nan     0.3896   -0.1291
##   2680       38.2716             nan     0.3896   -0.0846
##   2700       38.1455             nan     0.3896   -0.1166
##   2720       38.0508             nan     0.3896   -0.1213
##   2740       37.9538             nan     0.3896   -0.1311
##   2760       37.8647             nan     0.3896   -0.0672
##   2780       37.6963             nan     0.3896   -0.0420
##   2800       37.6276             nan     0.3896   -0.1571
##   2820       37.5078             nan     0.3896   -0.1000
##   2840       37.3660             nan     0.3896   -0.1027
##   2860       37.3271             nan     0.3896   -0.0987
##   2880       37.2108             nan     0.3896   -0.1287
##   2900       37.1023             nan     0.3896   -0.0733
##   2920       36.9750             nan     0.3896   -0.0977
##   2940       36.8505             nan     0.3896   -0.1107
##   2960       36.8358             nan     0.3896   -0.1205
##   2980       36.7271             nan     0.3896   -0.0633
##   3000       36.6034             nan     0.3896   -0.1032
##   3020       36.5134             nan     0.3896   -0.0617
##   3040       36.4105             nan     0.3896   -0.0785
##   3060       36.3294             nan     0.3896   -0.0572
##   3080       36.2050             nan     0.3896   -0.1324
##   3100       36.1944             nan     0.3896   -0.3713
##   3120       36.1019             nan     0.3896   -0.0473
##   3140       35.9728             nan     0.3896   -0.0953
##   3160       35.8641             nan     0.3896   -0.0999
##   3180       35.7981             nan     0.3896   -0.0864
##   3200       35.7123             nan     0.3896   -0.0912
##   3220       35.6955             nan     0.3896   -0.0850
##   3240       35.5785             nan     0.3896   -0.0639
##   3260       35.4647             nan     0.3896   -0.3206
##   3280       35.4006             nan     0.3896   -0.0325
##   3300       35.3258             nan     0.3896   -0.0454
##   3320       35.3097             nan     0.3896   -0.0410
##   3340       35.2573             nan     0.3896   -0.0743
##   3360       35.1747             nan     0.3896   -0.1555
##   3380       35.1081             nan     0.3896   -0.0566
##   3400       35.1279             nan     0.3896   -0.0905
##   3420       35.0406             nan     0.3896   -0.0422
##   3440       34.9268             nan     0.3896   -0.0547
##   3460       34.8506             nan     0.3896   -0.0650
##   3480       34.8232             nan     0.3896   -0.1557
##   3500       34.8129             nan     0.3896   -0.4201
##   3520       34.6681             nan     0.3896   -0.0877
##   3540       34.7046             nan     0.3896   -0.4916
##   3560       34.6142             nan     0.3896   -0.0295
##   3580       34.5245             nan     0.3896   -0.0654
##   3600       34.4668             nan     0.3896   -0.0863
##   3620       34.3957             nan     0.3896   -0.1030
##   3640       34.3557             nan     0.3896   -0.2238
##   3660       34.2600             nan     0.3896   -0.0848
##   3680       34.1688             nan     0.3896   -0.0413
##   3700       34.1401             nan     0.3896   -0.1209
##   3720       34.0873             nan     0.3896   -0.0301
##   3740       34.0117             nan     0.3896   -0.0984
##   3760       33.9812             nan     0.3896    0.0022
##   3780       33.8756             nan     0.3896   -0.0920
##   3800       33.7887             nan     0.3896   -0.0653
##   3820       33.7313             nan     0.3896   -0.0637
##   3840       33.6819             nan     0.3896   -0.1124
##   3860       33.6066             nan     0.3896   -0.0679
##   3880       33.5754             nan     0.3896   -0.0418
##   3900       33.5195             nan     0.3896   -0.0609
##   3920       33.4548             nan     0.3896   -0.1377
##   3940       33.4382             nan     0.3896   -0.0957
##   3960       33.3645             nan     0.3896   -0.1016
##   3980       33.2662             nan     0.3896   -0.0530
##   4000       33.2574             nan     0.3896   -0.0898
##   4020       33.1662             nan     0.3896   -0.0525
##   4040       33.1347             nan     0.3896   -0.0675
##   4060       33.0526             nan     0.3896   -0.0353
##   4080       32.9741             nan     0.3896   -0.0971
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      551.9731             nan     0.5470  638.5965
##      2      381.1555             nan     0.5470  164.1417
##      3      315.8714             nan     0.5470   61.7841
##      4      287.3096             nan     0.5470   28.0545
##      5      275.1857             nan     0.5470   11.5459
##      6      260.4648             nan     0.5470   13.0979
##      7      251.7576             nan     0.5470    7.2453
##      8      245.4612             nan     0.5470    5.5279
##      9      239.0924             nan     0.5470    4.9987
##     10      232.5708             nan     0.5470    5.4367
##     20      194.3271             nan     0.5470    3.1016
##     40      164.6481             nan     0.5470    0.8264
##     60      146.9274             nan     0.5470   -0.3204
##     80      133.5491             nan     0.5470    0.0071
##    100      125.9370             nan     0.5470   -0.0152
##    120      118.3852             nan     0.5470   -0.1809
##    140      113.2273             nan     0.5470   -0.4388
##    160      108.4859             nan     0.5470   -0.0915
##    180      103.6084             nan     0.5470   -0.3237
##    200       99.7271             nan     0.5470   -0.2485
##    220       96.5349             nan     0.5470   -0.2696
##    240       94.1205             nan     0.5470   -0.4432
##    260       91.3689             nan     0.5470   -0.1258
##    280       88.6295             nan     0.5470   -0.1268
##    300       86.6224             nan     0.5470   -0.2565
##    320       84.4482             nan     0.5470   -0.2164
##    340       82.4291             nan     0.5470   -0.1545
##    360       80.4927             nan     0.5470   -0.2878
##    380       78.6497             nan     0.5470   -0.1883
##    400       77.5114             nan     0.5470   -0.1680
##    420       76.2015             nan     0.5470   -0.2295
##    440       74.9412             nan     0.5470   -0.1248
##    460       73.7550             nan     0.5470   -0.4878
##    480       72.5179             nan     0.5470   -0.1957
##    500       71.7275             nan     0.5470   -0.4899
##    520       70.6845             nan     0.5470   -0.3163
##    540       69.6923             nan     0.5470   -0.1638
##    560       68.4473             nan     0.5470   -0.1358
##    580       67.7863             nan     0.5470   -0.2149
##    600       67.0501             nan     0.5470   -0.0414
##    620       66.1772             nan     0.5470   -0.4016
##    640       65.3527             nan     0.5470   -0.1379
##    660       64.5558             nan     0.5470   -0.1046
##    680       63.8072             nan     0.5470    0.0027
##    700       62.8732             nan     0.5470   -0.1552
##    720       62.3283             nan     0.5470   -0.2053
##    740       61.8211             nan     0.5470   -0.4524
##    760       61.1574             nan     0.5470   -0.1339
##    780       60.4313             nan     0.5470   -0.1831
##    800       59.8244             nan     0.5470   -0.1173
##    820       59.1227             nan     0.5470   -0.0616
##    840       58.4032             nan     0.5470   -0.0936
##    860       57.9002             nan     0.5470   -0.2349
##    880       57.0831             nan     0.5470   -0.0592
##    900       56.7251             nan     0.5470   -0.3941
##    920       56.0808             nan     0.5470   -0.1098
##    940       55.3472             nan     0.5470   -0.1270
##    960       55.1568             nan     0.5470   -0.0940
##    980       54.7041             nan     0.5470   -0.1750
##   1000       54.3076             nan     0.5470   -0.1086
##   1020       53.8875             nan     0.5470   -0.1628
##   1040       53.4368             nan     0.5470   -0.0800
##   1060       53.1067             nan     0.5470   -0.2687
##   1080       52.6952             nan     0.5470   -0.1260
##   1100       52.3135             nan     0.5470   -0.1394
##   1120       51.9353             nan     0.5470   -0.1390
##   1140       51.6637             nan     0.5470    0.0117
##   1160       51.4217             nan     0.5470   -0.1881
##   1180       51.0632             nan     0.5470   -0.0363
##   1200       50.7004             nan     0.5470   -0.1361
##   1220       50.3957             nan     0.5470   -0.1643
##   1240       49.9541             nan     0.5470   -0.2045
##   1260       49.7184             nan     0.5470   -0.0926
##   1280       49.4778             nan     0.5470   -0.2371
##   1300       49.2324             nan     0.5470   -0.2377
##   1320       48.8587             nan     0.5470   -0.2615
##   1340       48.5517             nan     0.5470   -0.2015
##   1360       48.2153             nan     0.5470   -0.1236
##   1380       47.9920             nan     0.5470   -0.1454
##   1400       47.6195             nan     0.5470   -0.0642
##   1420       47.5009             nan     0.5470   -0.0489
##   1440       47.4044             nan     0.5470   -0.0808
##   1460       47.0660             nan     0.5470   -0.2272
##   1480       46.9046             nan     0.5470   -0.1763
##   1500       46.5563             nan     0.5470   -0.1181
##   1520       46.2395             nan     0.5470   -0.1260
##   1540       45.9403             nan     0.5470   -0.2020
##   1560       45.7985             nan     0.5470   -0.3567
##   1580       45.4723             nan     0.5470   -0.1314
##   1600       45.3330             nan     0.5470   -0.2233
##   1620       45.0521             nan     0.5470   -0.1624
##   1640       44.8346             nan     0.5470   -0.1625
##   1660       44.4841             nan     0.5470   -0.1541
##   1680       44.3107             nan     0.5470   -0.2384
##   1700       44.1323             nan     0.5470   -0.1604
##   1720       43.9816             nan     0.5470   -0.1296
##   1740       43.8142             nan     0.5470   -0.0477
##   1760       43.8027             nan     0.5470   -0.6810
##   1780       43.5847             nan     0.5470   -0.1131
##   1800       43.5105             nan     0.5470   -0.1025
##   1820       43.4377             nan     0.5470   -0.3473
##   1840       43.2952             nan     0.5470   -0.2029
##   1860       42.8899             nan     0.5470   -0.0207
##   1880       42.6106             nan     0.5470   -0.1103
##   1900       42.3187             nan     0.5470   -0.2466
##   1920       42.0889             nan     0.5470   -0.0715
##   1940       41.8960             nan     0.5470   -0.2247
##   1960       41.7147             nan     0.5470   -0.1536
##   1980       41.5955             nan     0.5470   -0.1095
##   2000       41.4141             nan     0.5470   -0.1985
##   2020       41.2675             nan     0.5470   -0.1558
##   2040       41.1475             nan     0.5470   -0.0928
##   2060       40.9951             nan     0.5470   -0.1342
##   2080       40.8058             nan     0.5470   -0.1443
##   2100       40.6872             nan     0.5470   -0.1671
##   2120       40.5094             nan     0.5470   -0.0588
##   2140       40.4106             nan     0.5470   -0.0386
##   2160       40.3516             nan     0.5470   -0.2805
##   2180       40.2182             nan     0.5470   -0.1903
##   2200       40.1394             nan     0.5470   -0.2981
##   2220       39.9887             nan     0.5470   -0.1629
##   2240       39.8891             nan     0.5470   -0.1952
##   2260       39.7712             nan     0.5470   -0.2200
##   2280       39.6615             nan     0.5470   -0.1521
##   2300       39.5066             nan     0.5470   -0.0853
##   2320       39.4965             nan     0.5470   -0.1469
##   2340       39.3466             nan     0.5470   -0.0986
##   2360       39.2249             nan     0.5470   -0.0744
##   2380       39.2696             nan     0.5470   -0.2722
##   2400       39.0490             nan     0.5470   -0.1614
##   2420       38.9093             nan     0.5470   -0.1274
##   2440       38.9146             nan     0.5470   -0.1227
##   2460       38.8017             nan     0.5470   -0.1196
##   2480       38.6315             nan     0.5470   -0.1009
##   2500       38.5830             nan     0.5470   -0.2261
##   2520       38.4996             nan     0.5470   -0.2246
##   2540       38.3495             nan     0.5470   -0.0943
##   2560       38.2436             nan     0.5470   -0.3023
##   2580       38.2413             nan     0.5470   -0.2834
##   2600       38.1129             nan     0.5470   -0.3221
##   2620       37.9623             nan     0.5470   -0.1470
##   2640       37.8007             nan     0.5470   -0.2301
##   2660       37.8060             nan     0.5470   -0.3747
##   2680       37.7995             nan     0.5470   -0.1252
##   2700       37.7880             nan     0.5470   -0.5029
##   2720       37.6653             nan     0.5470   -0.1357
##   2740       37.5853             nan     0.5470   -0.1693
##   2760       37.3747             nan     0.5470   -0.1392
##   2780       37.2957             nan     0.5470   -0.1396
##   2800       37.3461             nan     0.5470   -0.2411
##   2820       37.2597             nan     0.5470   -0.1097
##   2840       37.1985             nan     0.5470   -0.2134
##   2860       37.0586             nan     0.5470   -0.0822
##   2880       36.9963             nan     0.5470   -0.2091
##   2900       36.9203             nan     0.5470   -0.1675
##   2920       36.8555             nan     0.5470   -0.1453
##   2940       36.7506             nan     0.5470   -0.0930
##   2960       36.6430             nan     0.5470   -0.0885
##   2980       36.7613             nan     0.5470   -0.0919
##   3000       36.6627             nan     0.5470   -0.1375
##   3020       36.5344             nan     0.5470   -0.1845
##   3040       36.5405             nan     0.5470   -0.1252
##   3060       36.5445             nan     0.5470   -0.2860
##   3080       36.4132             nan     0.5470   -0.1219
##   3100       36.2556             nan     0.5470   -0.0779
##   3120       36.3051             nan     0.5470   -0.1318
##   3140       36.2897             nan     0.5470   -0.1977
##   3160       36.2187             nan     0.5470   -0.0748
##   3180       36.0807             nan     0.5470   -0.1682
##   3200       35.9852             nan     0.5470   -0.0433
##   3220       36.0114             nan     0.5470   -0.2199
##   3240       35.8993             nan     0.5470   -0.1885
##   3260       35.8437             nan     0.5470   -0.1442
##   3280       35.7896             nan     0.5470   -0.0613
##   3300       35.7146             nan     0.5470   -0.1829
##   3320       35.5908             nan     0.5470   -0.1382
##   3340       35.5610             nan     0.5470   -0.1563
##   3360       35.5262             nan     0.5470   -0.2959
##   3380       35.3544             nan     0.5470   -0.0131
##   3400       35.3355             nan     0.5470   -0.1116
##   3420       35.2643             nan     0.5470   -0.1664
##   3440       35.1757             nan     0.5470   -0.1161
##   3460       35.0879             nan     0.5470   -0.1496
##   3480       35.0917             nan     0.5470   -0.1281
##   3489       35.0733             nan     0.5470   -0.1099
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      824.5032             nan     0.2306  358.7594
##      2      602.8762             nan     0.2306  218.3947
##      3      464.0081             nan     0.2306  139.9403
##      4      373.1170             nan     0.2306   87.2792
##      5      314.6836             nan     0.2306   56.7475
##      6      275.4414             nan     0.2306   37.9960
##      7      248.6566             nan     0.2306   25.9397
##      8      230.2933             nan     0.2306   17.7199
##      9      216.9252             nan     0.2306   11.9727
##     10      206.7317             nan     0.2306    9.3349
##     20      163.8796             nan     0.2306    1.7141
##     40      133.0524             nan     0.2306    0.7639
##     60      116.7856             nan     0.2306    0.3315
##     80      106.4404             nan     0.2306    0.0371
##    100       97.3825             nan     0.2306    0.0168
##    120       90.3686             nan     0.2306   -0.1658
##    140       84.9621             nan     0.2306   -0.0848
##    160       79.9917             nan     0.2306   -0.0244
##    180       76.1899             nan     0.2306   -0.1521
##    200       72.1666             nan     0.2306   -0.1482
##    220       69.0918             nan     0.2306   -0.1621
##    240       66.3786             nan     0.2306   -0.2010
##    260       63.6474             nan     0.2306   -0.1207
##    280       61.1302             nan     0.2306   -0.1167
##    300       58.8473             nan     0.2306   -0.0896
##    320       57.0501             nan     0.2306   -0.2334
##    340       55.3198             nan     0.2306   -0.0880
##    360       53.7088             nan     0.2306   -0.1095
##    380       52.1479             nan     0.2306   -0.3058
##    400       50.3986             nan     0.2306   -0.2488
##    420       49.1037             nan     0.2306   -0.1572
##    440       48.0553             nan     0.2306   -0.1990
##    460       46.9009             nan     0.2306   -0.2479
##    480       45.9653             nan     0.2306   -0.1983
##    500       44.9914             nan     0.2306   -0.1008
##    520       44.1757             nan     0.2306   -0.1013
##    540       43.2599             nan     0.2306   -0.1060
##    560       42.4597             nan     0.2306   -0.2213
##    580       41.7980             nan     0.2306   -0.2266
##    600       41.1184             nan     0.2306   -0.3225
##    620       40.3244             nan     0.2306   -0.2067
##    640       39.7375             nan     0.2306   -0.1362
##    660       39.2380             nan     0.2306   -0.1679
##    680       38.5639             nan     0.2306   -0.1238
##    700       38.1501             nan     0.2306   -0.1340
##    720       37.5051             nan     0.2306   -0.0775
##    740       37.0586             nan     0.2306   -0.1470
##    760       36.7057             nan     0.2306   -0.1541
##    780       36.2024             nan     0.2306   -0.1019
##    800       35.8285             nan     0.2306   -0.0979
##    820       35.4177             nan     0.2306   -0.1742
##    840       34.9678             nan     0.2306   -0.0896
##    860       34.5419             nan     0.2306   -0.2280
##    880       34.1168             nan     0.2306   -0.1592
##    900       33.7109             nan     0.2306   -0.1570
##    920       33.3359             nan     0.2306   -0.0764
##    940       32.9939             nan     0.2306   -0.1531
##    960       32.7770             nan     0.2306   -0.0818
##    980       32.3704             nan     0.2306   -0.0311
##   1000       32.1364             nan     0.2306   -0.1393
##   1020       31.8902             nan     0.2306   -0.1330
##   1040       31.6040             nan     0.2306   -0.1325
##   1060       31.4326             nan     0.2306   -0.2165
##   1080       31.2036             nan     0.2306   -0.1640
##   1100       30.9305             nan     0.2306   -0.1841
##   1120       30.7497             nan     0.2306   -0.1274
##   1140       30.5859             nan     0.2306   -0.1041
##   1160       30.4050             nan     0.2306   -0.1063
##   1180       30.2025             nan     0.2306   -0.1692
##   1200       30.0246             nan     0.2306   -0.1688
##   1220       29.8867             nan     0.2306   -0.1931
##   1240       29.6317             nan     0.2306   -0.0471
##   1260       29.4578             nan     0.2306   -0.1213
##   1280       29.3360             nan     0.2306   -0.1516
##   1300       29.2298             nan     0.2306   -0.0640
##   1320       29.0652             nan     0.2306   -0.1261
##   1340       28.9356             nan     0.2306   -0.2399
##   1360       28.7919             nan     0.2306   -0.1077
##   1380       28.6566             nan     0.2306   -0.1116
##   1400       28.5466             nan     0.2306   -0.1442
##   1420       28.3952             nan     0.2306   -0.1197
##   1440       28.3147             nan     0.2306   -0.2139
##   1460       28.1220             nan     0.2306   -0.1787
##   1480       27.9224             nan     0.2306   -0.1238
##   1500       27.7999             nan     0.2306   -0.1230
##   1520       27.6762             nan     0.2306   -0.1884
##   1540       27.5669             nan     0.2306   -0.1186
##   1560       27.4850             nan     0.2306   -0.1428
##   1580       27.3843             nan     0.2306   -0.0660
##   1600       27.3056             nan     0.2306   -0.0864
##   1620       27.1786             nan     0.2306   -0.1350
##   1640       27.0494             nan     0.2306   -0.0375
##   1660       27.0183             nan     0.2306   -0.1578
##   1680       26.9686             nan     0.2306   -0.1544
##   1700       26.8209             nan     0.2306   -0.2855
##   1720       26.6955             nan     0.2306   -0.1143
##   1740       26.5769             nan     0.2306   -0.1382
##   1760       26.4889             nan     0.2306   -0.1159
##   1780       26.4049             nan     0.2306   -0.0947
##   1800       26.3146             nan     0.2306   -0.0891
##   1820       26.2605             nan     0.2306   -0.0572
##   1840       26.2008             nan     0.2306   -0.1500
##   1860       26.1285             nan     0.2306   -0.1846
##   1880       26.1385             nan     0.2306   -0.2176
##   1900       25.9891             nan     0.2306   -0.1341
##   1920       25.9143             nan     0.2306   -0.1143
##   1940       25.8488             nan     0.2306   -0.0886
##   1960       25.7927             nan     0.2306   -0.1476
##   1980       25.7497             nan     0.2306   -0.1884
##   2000       25.6909             nan     0.2306   -0.1705
##   2020       25.5852             nan     0.2306   -0.0803
##   2040       25.4981             nan     0.2306   -0.1208
##   2060       25.4482             nan     0.2306   -0.1295
##   2080       25.3058             nan     0.2306   -0.1088
##   2100       25.2177             nan     0.2306   -0.0869
##   2120       25.1747             nan     0.2306   -0.0986
##   2140       25.1074             nan     0.2306   -0.0651
##   2160       25.0492             nan     0.2306   -0.0526
##   2180       24.9790             nan     0.2306   -0.0638
##   2200       24.9555             nan     0.2306   -0.0537
##   2220       24.8798             nan     0.2306   -0.0514
##   2240       24.7932             nan     0.2306   -0.0948
##   2260       24.7678             nan     0.2306   -0.1029
##   2280       24.7128             nan     0.2306   -0.0798
##   2300       24.6902             nan     0.2306   -0.1464
##   2320       24.6225             nan     0.2306   -0.0694
##   2340       24.6463             nan     0.2306   -0.0760
##   2360       24.5509             nan     0.2306   -0.2214
##   2380       24.4958             nan     0.2306   -0.1149
##   2400       24.4500             nan     0.2306   -0.0977
##   2420       24.4441             nan     0.2306   -0.1574
##   2440       24.3766             nan     0.2306   -0.1606
##   2460       24.2865             nan     0.2306   -0.0496
##   2480       24.2534             nan     0.2306   -0.0645
##   2500       24.1952             nan     0.2306   -0.1443
##   2520       24.1762             nan     0.2306   -0.1407
##   2540       24.1635             nan     0.2306   -0.1334
##   2560       24.1188             nan     0.2306   -0.1231
##   2580       24.1100             nan     0.2306   -0.0878
##   2600       24.0546             nan     0.2306   -0.1168
##   2620       24.0522             nan     0.2306   -0.1046
##   2640       23.9633             nan     0.2306   -0.2214
##   2660       23.9170             nan     0.2306   -0.1343
##   2680       23.8495             nan     0.2306   -0.0995
##   2700       23.7887             nan     0.2306   -0.0992
##   2720       23.7930             nan     0.2306   -0.1271
##   2740       23.7055             nan     0.2306   -0.1114
##   2760       23.7048             nan     0.2306   -0.1145
##   2780       23.6803             nan     0.2306   -0.2032
##   2800       23.6571             nan     0.2306   -0.1405
##   2820       23.6065             nan     0.2306   -0.2271
##   2840       23.5749             nan     0.2306   -0.0958
##   2860       23.5475             nan     0.2306   -0.1426
##   2880       23.5379             nan     0.2306   -0.1520
##   2900       23.4992             nan     0.2306   -0.1104
##   2920       23.4745             nan     0.2306   -0.0425
##   2940       23.4463             nan     0.2306   -0.1049
##   2960       23.4108             nan     0.2306   -0.0249
##   2980       23.3516             nan     0.2306   -0.0429
##   3000       23.3262             nan     0.2306   -0.0776
##   3020       23.3207             nan     0.2306   -0.1422
##   3040       23.2846             nan     0.2306   -0.1803
##   3060       23.2596             nan     0.2306   -0.1003
##   3080       23.2887             nan     0.2306   -0.1694
##   3100       23.2525             nan     0.2306   -0.1736
##   3120       23.1977             nan     0.2306   -0.1343
##   3140       23.2284             nan     0.2306   -0.0869
##   3160       23.1945             nan     0.2306   -0.2439
##   3180       23.1641             nan     0.2306   -0.2110
##   3200       23.0972             nan     0.2306   -0.1107
##   3220       23.0840             nan     0.2306   -0.1662
##   3240       23.0224             nan     0.2306   -0.1066
##   3260       23.0106             nan     0.2306   -0.2042
##   3280       23.0024             nan     0.2306   -0.1358
##   3300       22.9757             nan     0.2306   -0.1037
##   3320       22.9466             nan     0.2306   -0.0573
##   3340       22.8861             nan     0.2306   -0.0984
##   3360       22.8520             nan     0.2306   -0.1346
##   3380       22.8440             nan     0.2306   -0.1262
##   3400       22.8351             nan     0.2306   -0.1342
##   3420       22.7839             nan     0.2306   -0.1059
##   3440       22.8117             nan     0.2306   -0.1889
##   3460       22.7635             nan     0.2306   -0.1571
##   3480       22.7803             nan     0.2306   -0.1192
##   3500       22.7506             nan     0.2306   -0.0922
##   3520       22.7167             nan     0.2306   -0.1205
##   3540       22.7646             nan     0.2306   -0.3213
##   3560       22.7455             nan     0.2306   -0.1089
##   3580       22.6815             nan     0.2306   -0.0948
##   3600       22.6599             nan     0.2306   -0.1139
##   3620       22.6339             nan     0.2306   -0.1306
##   3640       22.5907             nan     0.2306   -0.0924
##   3660       22.5897             nan     0.2306   -0.0755
##   3680       22.5878             nan     0.2306   -0.0389
##   3700       22.5453             nan     0.2306   -0.0883
##   3720       22.5001             nan     0.2306   -0.0678
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      676.7874             nan     0.3896  503.0298
##      2      462.1272             nan     0.3896  217.5228
##      3      369.3819             nan     0.3896   91.6676
##      4      321.1326             nan     0.3896   48.9746
##      5      292.0548             nan     0.3896   29.0610
##      6      276.6196             nan     0.3896   15.4489
##      7      267.1979             nan     0.3896    8.8064
##      8      256.3657             nan     0.3896   10.2001
##      9      249.4938             nan     0.3896    6.1156
##     10      241.7859             nan     0.3896    7.3177
##     20      204.6758             nan     0.3896    1.9954
##     40      171.9567             nan     0.3896    0.6936
##     60      154.4838             nan     0.3896    0.1518
##     80      141.4603             nan     0.3896    0.1641
##    100      131.6954             nan     0.3896   -0.1131
##    120      124.0250             nan     0.3896   -0.0188
##    140      118.8261             nan     0.3896   -0.0867
##    160      113.3610             nan     0.3896   -0.0144
##    180      109.7572             nan     0.3896   -0.1600
##    200      105.9059             nan     0.3896   -0.3059
##    220      101.8396             nan     0.3896   -0.1546
##    240       97.9844             nan     0.3896    0.0017
##    260       95.0951             nan     0.3896   -0.0098
##    280       92.9656             nan     0.3896   -0.1020
##    300       91.5154             nan     0.3896   -0.1820
##    320       89.6842             nan     0.3896   -0.0738
##    340       87.4336             nan     0.3896   -0.1016
##    360       85.6375             nan     0.3896   -0.0074
##    380       83.5569             nan     0.3896   -0.0381
##    400       81.5011             nan     0.3896   -0.1940
##    420       80.2534             nan     0.3896   -0.2900
##    440       78.9520             nan     0.3896   -0.3901
##    460       77.2449             nan     0.3896   -0.1287
##    480       75.7496             nan     0.3896   -0.1397
##    500       74.6485             nan     0.3896   -0.0332
##    520       73.4894             nan     0.3896   -0.1106
##    540       72.4329             nan     0.3896   -0.2478
##    560       71.5887             nan     0.3896   -0.1621
##    580       70.8836             nan     0.3896   -0.2270
##    600       69.9890             nan     0.3896   -0.1757
##    620       69.0725             nan     0.3896   -0.0638
##    640       68.0818             nan     0.3896   -0.0799
##    660       67.1039             nan     0.3896   -0.0329
##    680       66.4275             nan     0.3896   -0.2072
##    700       65.6793             nan     0.3896   -0.2348
##    720       64.7027             nan     0.3896   -0.1839
##    740       64.1165             nan     0.3896   -0.0104
##    760       63.4867             nan     0.3896   -0.0646
##    780       62.8271             nan     0.3896   -0.0672
##    800       62.1435             nan     0.3896   -0.1374
##    820       61.5516             nan     0.3896   -0.1716
##    840       61.1191             nan     0.3896   -0.2039
##    860       60.5658             nan     0.3896   -0.1957
##    880       60.1272             nan     0.3896   -0.0647
##    900       59.6326             nan     0.3896   -0.0670
##    920       59.2187             nan     0.3896   -0.0671
##    940       58.7571             nan     0.3896   -0.0759
##    960       58.3047             nan     0.3896   -0.0923
##    980       57.7488             nan     0.3896   -0.1091
##   1000       57.1954             nan     0.3896   -0.0849
##   1020       56.6139             nan     0.3896   -0.0456
##   1040       56.1948             nan     0.3896   -0.0731
##   1060       55.7054             nan     0.3896   -0.1210
##   1080       55.1929             nan     0.3896   -0.1745
##   1100       54.7042             nan     0.3896   -0.0754
##   1120       54.3278             nan     0.3896   -0.1089
##   1140       53.9743             nan     0.3896   -0.1038
##   1160       53.5408             nan     0.3896   -0.1734
##   1180       53.1545             nan     0.3896   -0.1132
##   1200       52.7538             nan     0.3896   -0.1956
##   1220       52.4848             nan     0.3896   -0.1043
##   1240       52.1655             nan     0.3896   -0.1915
##   1260       51.8808             nan     0.3896   -0.0835
##   1280       51.4990             nan     0.3896   -0.0639
##   1300       51.0864             nan     0.3896   -0.1306
##   1320       50.8608             nan     0.3896   -0.0614
##   1340       50.4388             nan     0.3896   -0.0872
##   1360       50.0544             nan     0.3896   -0.1688
##   1380       49.8039             nan     0.3896   -0.0831
##   1400       49.3310             nan     0.3896   -0.0684
##   1420       49.0754             nan     0.3896   -0.0810
##   1440       48.5394             nan     0.3896   -0.1353
##   1460       48.2373             nan     0.3896   -0.0448
##   1480       47.8167             nan     0.3896   -0.0651
##   1500       47.4877             nan     0.3896   -0.1033
##   1520       47.1899             nan     0.3896   -0.1368
##   1540       46.9605             nan     0.3896   -0.1168
##   1560       46.7671             nan     0.3896   -0.1675
##   1580       46.5418             nan     0.3896   -0.1274
##   1600       46.2932             nan     0.3896   -0.0774
##   1620       45.9785             nan     0.3896   -0.0975
##   1640       45.7235             nan     0.3896   -0.1089
##   1660       45.4349             nan     0.3896   -0.0717
##   1680       45.2012             nan     0.3896   -0.1227
##   1700       45.0619             nan     0.3896   -0.0673
##   1720       44.8754             nan     0.3896   -0.0701
##   1740       44.5822             nan     0.3896   -0.0016
##   1760       44.4372             nan     0.3896   -0.2114
##   1780       44.2204             nan     0.3896   -0.0916
##   1800       44.0977             nan     0.3896   -0.1041
##   1820       43.8926             nan     0.3896   -0.0844
##   1840       43.7616             nan     0.3896   -0.1153
##   1860       43.4829             nan     0.3896   -0.1127
##   1880       43.3235             nan     0.3896   -0.0672
##   1900       43.0746             nan     0.3896   -0.0592
##   1920       42.8841             nan     0.3896   -0.0877
##   1940       42.6049             nan     0.3896   -0.1000
##   1960       42.3771             nan     0.3896   -0.0960
##   1980       42.2022             nan     0.3896   -0.1194
##   2000       42.0165             nan     0.3896   -0.0727
##   2020       41.9102             nan     0.3896   -0.0669
##   2040       41.7578             nan     0.3896   -0.0751
##   2060       41.6062             nan     0.3896   -0.1005
##   2080       41.4870             nan     0.3896   -0.0751
##   2100       41.3008             nan     0.3896   -0.1438
##   2120       41.2152             nan     0.3896   -0.0725
##   2140       41.0719             nan     0.3896   -0.0947
##   2160       40.9667             nan     0.3896   -0.1601
##   2180       40.8411             nan     0.3896   -0.0374
##   2200       40.7262             nan     0.3896   -0.1341
##   2220       40.6091             nan     0.3896   -0.2143
##   2240       40.4798             nan     0.3896   -0.0928
##   2260       40.2775             nan     0.3896   -0.1040
##   2280       40.1822             nan     0.3896   -0.1540
##   2300       40.0495             nan     0.3896   -0.0409
##   2320       39.8311             nan     0.3896   -0.2209
##   2340       39.7834             nan     0.3896   -0.1052
##   2360       39.6512             nan     0.3896   -0.1208
##   2380       39.4127             nan     0.3896   -0.0611
##   2400       39.3839             nan     0.3896   -0.1135
##   2420       39.2398             nan     0.3896   -0.0600
##   2440       39.0575             nan     0.3896   -0.0679
##   2460       38.9582             nan     0.3896   -0.0562
##   2480       38.8427             nan     0.3896   -0.0873
##   2500       38.8203             nan     0.3896   -0.1018
##   2520       38.6111             nan     0.3896   -0.0779
##   2540       38.5028             nan     0.3896   -0.1741
##   2560       38.3847             nan     0.3896   -0.0558
##   2580       38.2446             nan     0.3896   -0.0542
##   2600       38.1379             nan     0.3896   -0.1146
##   2620       37.9773             nan     0.3896   -0.1078
##   2640       37.8399             nan     0.3896   -0.0330
##   2660       37.6682             nan     0.3896   -0.0808
##   2680       37.5244             nan     0.3896   -0.1111
##   2700       37.4741             nan     0.3896   -0.0376
##   2720       37.3291             nan     0.3896   -0.0754
##   2740       37.1783             nan     0.3896   -0.1129
##   2760       37.0910             nan     0.3896   -0.0882
##   2780       37.0480             nan     0.3896   -0.0953
##   2800       36.8962             nan     0.3896   -0.1007
##   2820       36.7865             nan     0.3896   -0.0665
##   2840       36.6619             nan     0.3896   -0.0681
##   2860       36.6333             nan     0.3896   -0.0634
##   2880       36.5219             nan     0.3896   -0.1025
##   2900       36.4865             nan     0.3896   -0.0564
##   2920       36.4256             nan     0.3896   -0.1745
##   2940       36.3368             nan     0.3896   -0.1579
##   2960       36.2459             nan     0.3896   -0.1270
##   2980       36.1531             nan     0.3896   -0.1513
##   3000       36.0341             nan     0.3896   -0.1159
##   3020       35.9299             nan     0.3896   -0.0661
##   3040       35.8598             nan     0.3896   -0.0791
##   3060       35.7775             nan     0.3896   -0.1142
##   3080       35.6993             nan     0.3896   -0.0720
##   3100       35.6068             nan     0.3896   -0.1135
##   3120       35.5469             nan     0.3896   -0.1477
##   3140       35.4928             nan     0.3896   -0.0746
##   3160       35.4537             nan     0.3896   -0.1369
##   3180       35.3915             nan     0.3896   -0.0891
##   3200       35.2540             nan     0.3896   -0.1028
##   3220       35.1759             nan     0.3896   -0.1115
##   3240       35.0391             nan     0.3896   -0.0677
##   3260       34.9902             nan     0.3896   -0.1324
##   3280       34.9695             nan     0.3896   -0.0647
##   3300       34.8306             nan     0.3896   -0.0895
##   3320       34.7354             nan     0.3896   -0.0433
##   3340       34.7117             nan     0.3896   -0.2067
##   3360       34.6097             nan     0.3896   -0.0957
##   3380       34.5283             nan     0.3896   -0.1152
##   3400       34.4839             nan     0.3896   -0.1711
##   3420       34.3977             nan     0.3896   -0.0642
##   3440       34.3447             nan     0.3896   -0.1031
##   3460       34.3113             nan     0.3896   -0.1636
##   3480       34.1802             nan     0.3896   -0.0445
##   3500       34.1486             nan     0.3896   -0.0365
##   3520       34.0548             nan     0.3896   -0.0754
##   3540       33.9576             nan     0.3896   -0.1056
##   3560       33.9306             nan     0.3896   -0.0829
##   3580       33.9437             nan     0.3896   -0.3327
##   3600       33.7716             nan     0.3896   -0.1081
##   3620       33.6881             nan     0.3896   -0.0497
##   3640       33.6302             nan     0.3896   -0.0662
##   3660       33.6233             nan     0.3896   -0.1971
##   3680       33.5466             nan     0.3896   -0.0810
##   3700       33.4634             nan     0.3896   -0.0638
##   3720       33.4878             nan     0.3896   -0.1659
##   3740       33.4348             nan     0.3896   -0.0971
##   3760       33.3554             nan     0.3896   -0.0899
##   3780       33.2806             nan     0.3896   -0.0373
##   3800       33.2320             nan     0.3896   -0.0573
##   3820       33.2227             nan     0.3896   -0.0436
##   3840       33.1606             nan     0.3896   -0.1817
##   3860       33.0350             nan     0.3896   -0.0389
##   3880       32.9366             nan     0.3896   -0.1747
##   3900       32.8574             nan     0.3896   -0.0962
##   3920       32.8301             nan     0.3896   -0.0751
##   3940       32.7757             nan     0.3896   -0.0657
##   3960       32.6493             nan     0.3896   -0.0313
##   3980       32.6333             nan     0.3896   -0.0686
##   4000       32.5437             nan     0.3896   -0.1273
##   4020       32.5393             nan     0.3896   -0.2766
##   4040       32.4834             nan     0.3896   -0.1251
##   4060       32.4715             nan     0.3896   -0.0828
##   4080       32.3601             nan     0.3896   -0.0932
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      546.5661             nan     0.5470  641.0665
##      2      376.3570             nan     0.5470  170.3941
##      3      311.1084             nan     0.5470   64.0566
##      4      283.3075             nan     0.5470   22.7322
##      5      269.2665             nan     0.5470   12.1286
##      6      258.0019             nan     0.5470    9.3113
##      7      243.7206             nan     0.5470   13.7227
##      8      236.8729             nan     0.5470    5.9110
##      9      231.7542             nan     0.5470    4.1337
##     10      226.1930             nan     0.5470    5.1666
##     20      194.3862             nan     0.5470    1.9624
##     40      163.5186             nan     0.5470    0.9037
##     60      145.6874             nan     0.5470   -0.4706
##     80      136.6357             nan     0.5470   -0.8937
##    100      128.4355             nan     0.5470   -0.5505
##    120      122.2153             nan     0.5470   -0.2268
##    140      115.7605             nan     0.5470   -0.4969
##    160      111.0838             nan     0.5470   -0.0857
##    180      107.2105             nan     0.5470   -0.2006
##    200      102.9258             nan     0.5470   -0.2066
##    220      100.1367             nan     0.5470   -0.3513
##    240       97.2569             nan     0.5470   -0.2090
##    260       94.2185             nan     0.5470   -0.3163
##    280       92.2319             nan     0.5470   -0.2763
##    300       90.0157             nan     0.5470   -0.0852
##    320       87.7563             nan     0.5470   -0.2412
##    340       85.3624             nan     0.5470    0.0670
##    360       83.4321             nan     0.5470   -0.4163
##    380       81.7713             nan     0.5470   -0.3095
##    400       80.0384             nan     0.5470   -0.1843
##    420       78.4607             nan     0.5470   -0.2436
##    440       76.8827             nan     0.5470   -0.2332
##    460       75.5269             nan     0.5470   -0.1339
##    480       73.9243             nan     0.5470   -0.1011
##    500       72.5162             nan     0.5470   -0.1849
##    520       71.3064             nan     0.5470   -0.1424
##    540       70.2495             nan     0.5470   -0.2111
##    560       69.2767             nan     0.5470   -0.1808
##    580       68.2879             nan     0.5470   -0.3138
##    600       67.5192             nan     0.5470   -0.4171
##    620       66.5601             nan     0.5470   -0.2388
##    640       65.6637             nan     0.5470   -0.0992
##    660       64.7229             nan     0.5470   -0.2879
##    680       64.0665             nan     0.5470   -0.2142
##    700       63.3618             nan     0.5470   -0.0640
##    720       62.4720             nan     0.5470   -0.2646
##    740       61.6768             nan     0.5470   -0.1569
##    760       60.8921             nan     0.5470   -0.1717
##    780       60.1663             nan     0.5470   -0.2218
##    800       59.5106             nan     0.5470   -0.2020
##    820       58.8745             nan     0.5470   -0.0819
##    840       58.2365             nan     0.5470   -0.2009
##    860       57.7834             nan     0.5470   -0.5404
##    880       57.2998             nan     0.5470   -0.1650
##    900       56.7681             nan     0.5470   -0.0522
##    920       56.2528             nan     0.5470   -0.1455
##    940       56.0001             nan     0.5470   -0.2303
##    960       55.6396             nan     0.5470   -0.1041
##    980       55.2729             nan     0.5470   -0.1234
##   1000       54.9231             nan     0.5470   -0.2537
##   1020       54.5413             nan     0.5470   -0.2304
##   1040       54.0163             nan     0.5470   -0.1779
##   1060       53.6049             nan     0.5470   -0.1047
##   1080       53.1617             nan     0.5470   -0.0618
##   1100       52.8327             nan     0.5470   -0.2596
##   1120       52.4054             nan     0.5470   -0.0512
##   1140       52.0399             nan     0.5470   -0.1604
##   1160       51.7141             nan     0.5470   -0.2805
##   1180       51.3139             nan     0.5470   -0.2143
##   1200       50.9560             nan     0.5470   -0.2674
##   1220       50.7072             nan     0.5470   -0.2872
##   1240       50.2833             nan     0.5470   -0.1485
##   1260       49.9520             nan     0.5470   -0.2206
##   1280       49.5444             nan     0.5470   -0.1601
##   1300       49.2284             nan     0.5470   -0.0329
##   1320       48.8895             nan     0.5470   -0.1568
##   1340       48.5203             nan     0.5470   -0.1167
##   1360       48.2126             nan     0.5470   -0.1167
##   1380       47.9532             nan     0.5470   -0.1552
##   1400       47.6969             nan     0.5470   -0.1928
##   1420       47.4716             nan     0.5470   -0.2516
##   1440       47.2213             nan     0.5470   -0.2152
##   1460       47.1133             nan     0.5470   -0.2246
##   1480       46.7965             nan     0.5470   -0.0723
##   1500       46.6405             nan     0.5470   -0.2787
##   1520       46.2043             nan     0.5470   -0.1304
##   1540       46.0356             nan     0.5470   -0.3637
##   1560       45.8195             nan     0.5470   -0.0614
##   1580       45.7218             nan     0.5470   -0.6509
##   1600       45.4390             nan     0.5470   -0.1670
##   1620       45.2994             nan     0.5470   -0.2902
##   1640       44.9795             nan     0.5470   -0.1743
##   1660       44.8234             nan     0.5470   -0.2952
##   1680       44.4551             nan     0.5470   -0.1528
##   1700       44.3912             nan     0.5470   -0.3037
##   1720       44.0968             nan     0.5470   -0.1346
##   1740       43.9784             nan     0.5470   -0.2377
##   1760       43.7942             nan     0.5470   -0.2863
##   1780       43.5378             nan     0.5470   -0.0925
##   1800       43.3838             nan     0.5470   -0.1641
##   1820       43.1275             nan     0.5470   -0.0948
##   1840       42.9637             nan     0.5470   -0.1626
##   1860       42.7432             nan     0.5470   -0.0886
##   1880       42.5478             nan     0.5470   -0.1150
##   1900       42.3355             nan     0.5470   -0.0525
##   1920       42.2099             nan     0.5470   -0.1942
##   1940       42.0787             nan     0.5470   -0.1890
##   1960       41.9156             nan     0.5470   -0.1586
##   1980       41.8686             nan     0.5470   -0.2626
##   2000       41.8021             nan     0.5470   -0.4455
##   2020       41.5710             nan     0.5470   -0.1698
##   2040       41.3957             nan     0.5470   -0.1744
##   2060       41.2927             nan     0.5470   -0.1867
##   2080       41.1829             nan     0.5470   -0.2127
##   2100       40.9783             nan     0.5470   -0.1668
##   2120       40.6914             nan     0.5470   -0.0516
##   2140       40.5860             nan     0.5470   -0.1782
##   2160       40.4169             nan     0.5470   -0.1933
##   2180       40.4164             nan     0.5470   -0.4715
##   2200       40.1695             nan     0.5470   -0.1923
##   2220       40.1263             nan     0.5470   -0.1078
##   2240       40.0211             nan     0.5470   -0.1873
##   2260       39.8246             nan     0.5470   -0.1064
##   2280       39.6909             nan     0.5470   -0.1938
##   2300       39.5407             nan     0.5470   -0.1404
##   2320       39.4408             nan     0.5470   -0.1769
##   2340       39.3522             nan     0.5470   -0.1558
##   2360       39.2391             nan     0.5470   -0.1730
##   2380       39.2133             nan     0.5470   -0.0453
##   2400       39.1104             nan     0.5470   -0.2066
##   2420       39.0067             nan     0.5470   -0.0802
##   2440       38.7941             nan     0.5470   -0.1459
##   2460       38.5926             nan     0.5470   -0.1749
##   2480       38.6276             nan     0.5470   -0.2508
##   2500       38.4237             nan     0.5470   -0.1558
##   2520       38.3170             nan     0.5470   -0.1633
##   2540       38.1402             nan     0.5470   -0.2759
##   2560       38.0954             nan     0.5470   -0.0568
##   2580       37.9986             nan     0.5470   -0.0939
##   2600       38.0530             nan     0.5470   -0.5297
##   2620       37.7259             nan     0.5470   -0.1556
##   2640       37.6480             nan     0.5470   -0.1346
##   2660       37.5463             nan     0.5470   -0.1037
##   2680       37.4748             nan     0.5470   -0.4596
##   2700       37.3463             nan     0.5470   -0.1313
##   2720       37.2085             nan     0.5470   -0.0976
##   2740       37.0329             nan     0.5470   -0.1684
##   2760       36.9484             nan     0.5470   -0.0875
##   2780       36.8370             nan     0.5470   -0.1809
##   2800       36.7521             nan     0.5470   -0.2574
##   2820       36.8061             nan     0.5470   -0.1501
##   2840       36.7004             nan     0.5470   -0.1380
##   2860       36.5675             nan     0.5470   -0.0801
##   2880       36.4309             nan     0.5470   -0.0847
##   2900       36.3214             nan     0.5470   -0.1583
##   2920       36.2667             nan     0.5470   -0.2150
##   2940       36.1818             nan     0.5470   -0.3765
##   2960       36.0877             nan     0.5470   -0.2005
##   2980       36.0490             nan     0.5470   -0.1911
##   3000       36.0528             nan     0.5470   -0.1971
##   3020       35.7958             nan     0.5470   -0.0586
##   3040       35.6981             nan     0.5470   -0.0549
##   3060       35.7219             nan     0.5470   -0.1138
##   3080       35.7903             nan     0.5470   -0.4219
##   3100       35.5469             nan     0.5470   -0.1295
##   3120       35.4998             nan     0.5470   -0.1575
##   3140       35.4864             nan     0.5470    0.0087
##   3160       35.3572             nan     0.5470   -0.0387
##   3180       35.2754             nan     0.5470   -0.0959
##   3200       35.2579             nan     0.5470   -0.1290
##   3220       35.2564             nan     0.5470   -0.1930
##   3240       35.0663             nan     0.5470   -0.0564
##   3260       35.0350             nan     0.5470   -0.2138
##   3280       34.8091             nan     0.5470   -0.1120
##   3300       34.7420             nan     0.5470   -0.1413
##   3320       34.7197             nan     0.5470   -0.1303
##   3340       34.7141             nan     0.5470   -0.3466
##   3360       34.7008             nan     0.5470   -0.1359
##   3380       34.5466             nan     0.5470   -0.1431
##   3400       34.4990             nan     0.5470   -0.0861
##   3420       34.4874             nan     0.5470   -0.0457
##   3440       34.3689             nan     0.5470   -0.1197
##   3460       34.2587             nan     0.5470   -0.1860
##   3480       34.1296             nan     0.5470   -0.1261
##   3489       34.0941             nan     0.5470   -0.0940
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      824.5621             nan     0.2306  357.5419
##      2      606.2890             nan     0.2306  218.9355
##      3      464.9577             nan     0.2306  137.8700
##      4      376.8895             nan     0.2306   88.1293
##      5      319.7340             nan     0.2306   57.5879
##      6      280.9460             nan     0.2306   37.6326
##      7      252.7670             nan     0.2306   27.6333
##      8      234.4730             nan     0.2306   17.5855
##      9      221.0968             nan     0.2306   13.1766
##     10      210.9191             nan     0.2306    9.2725
##     20      167.0790             nan     0.2306    1.4132
##     40      134.3664             nan     0.2306    0.3136
##     60      116.9811             nan     0.2306    0.2405
##     80      106.6758             nan     0.2306   -0.2214
##    100       97.7058             nan     0.2306    0.0582
##    120       90.6339             nan     0.2306    0.0202
##    140       85.1634             nan     0.2306   -0.1643
##    160       79.9563             nan     0.2306   -0.2670
##    180       75.9831             nan     0.2306   -0.0709
##    200       72.1967             nan     0.2306   -0.1579
##    220       69.0412             nan     0.2306   -0.1711
##    240       66.0974             nan     0.2306   -0.1587
##    260       63.4029             nan     0.2306   -0.1677
##    280       61.0143             nan     0.2306   -0.2201
##    300       58.8483             nan     0.2306    0.0683
##    320       56.6593             nan     0.2306   -0.0757
##    340       55.0994             nan     0.2306   -0.1701
##    360       53.6138             nan     0.2306   -0.1535
##    380       52.1406             nan     0.2306   -0.1320
##    400       50.8933             nan     0.2306   -0.3028
##    420       49.7309             nan     0.2306   -0.3120
##    440       48.5677             nan     0.2306   -0.0999
##    460       47.2341             nan     0.2306   -0.1240
##    480       46.1920             nan     0.2306   -0.2287
##    500       45.1822             nan     0.2306   -0.2430
##    520       44.2399             nan     0.2306   -0.0932
##    540       43.4788             nan     0.2306   -0.1985
##    560       42.7071             nan     0.2306   -0.1199
##    580       41.8763             nan     0.2306   -0.0849
##    600       41.1899             nan     0.2306   -0.1755
##    620       40.6568             nan     0.2306   -0.1077
##    640       40.1204             nan     0.2306   -0.1063
##    660       39.5524             nan     0.2306   -0.1604
##    680       38.9617             nan     0.2306   -0.1168
##    700       38.5263             nan     0.2306   -0.1890
##    720       38.0407             nan     0.2306   -0.1895
##    740       37.6491             nan     0.2306   -0.1075
##    760       37.1249             nan     0.2306   -0.1657
##    780       36.7749             nan     0.2306   -0.1646
##    800       36.3994             nan     0.2306   -0.1476
##    820       36.0038             nan     0.2306   -0.1727
##    840       35.5173             nan     0.2306   -0.1111
##    860       35.1277             nan     0.2306   -0.1104
##    880       34.7810             nan     0.2306   -0.1301
##    900       34.5154             nan     0.2306   -0.1456
##    920       34.2253             nan     0.2306   -0.1148
##    940       33.8886             nan     0.2306   -0.1617
##    960       33.6057             nan     0.2306   -0.1763
##    980       33.3641             nan     0.2306   -0.1493
##   1000       33.0285             nan     0.2306   -0.1065
##   1020       32.8366             nan     0.2306   -0.2730
##   1040       32.5874             nan     0.2306   -0.1524
##   1060       32.3549             nan     0.2306   -0.3181
##   1080       32.0719             nan     0.2306   -0.0423
##   1100       31.8083             nan     0.2306   -0.0673
##   1120       31.5794             nan     0.2306   -0.1368
##   1140       31.3509             nan     0.2306   -0.0799
##   1160       31.1488             nan     0.2306   -0.1068
##   1180       30.9390             nan     0.2306   -0.1364
##   1200       30.7420             nan     0.2306   -0.1169
##   1220       30.5889             nan     0.2306   -0.1313
##   1240       30.3853             nan     0.2306   -0.1398
##   1260       30.1303             nan     0.2306   -0.0919
##   1280       29.8632             nan     0.2306   -0.0543
##   1300       29.7769             nan     0.2306   -0.1612
##   1320       29.5437             nan     0.2306   -0.1324
##   1340       29.4203             nan     0.2306   -0.1609
##   1360       29.2314             nan     0.2306   -0.2391
##   1380       29.0911             nan     0.2306   -0.0561
##   1400       28.9397             nan     0.2306   -0.0836
##   1420       28.8238             nan     0.2306   -0.1213
##   1440       28.6886             nan     0.2306   -0.1308
##   1460       28.5297             nan     0.2306   -0.1295
##   1480       28.4300             nan     0.2306   -0.1966
##   1500       28.3322             nan     0.2306   -0.0946
##   1520       28.2422             nan     0.2306   -0.1913
##   1540       28.1231             nan     0.2306   -0.1030
##   1560       28.0336             nan     0.2306   -0.1264
##   1580       27.9447             nan     0.2306   -0.1251
##   1600       27.8523             nan     0.2306   -0.1244
##   1620       27.7223             nan     0.2306   -0.1346
##   1640       27.6466             nan     0.2306   -0.1866
##   1660       27.5731             nan     0.2306   -0.1287
##   1680       27.4824             nan     0.2306   -0.1574
##   1700       27.4106             nan     0.2306   -0.1153
##   1720       27.3141             nan     0.2306   -0.0674
##   1740       27.2428             nan     0.2306   -0.1385
##   1760       27.0648             nan     0.2306   -0.0825
##   1780       27.0419             nan     0.2306   -0.2381
##   1800       26.9406             nan     0.2306   -0.0875
##   1820       26.8727             nan     0.2306   -0.1115
##   1840       26.7248             nan     0.2306   -0.1537
##   1860       26.6582             nan     0.2306   -0.1143
##   1880       26.5882             nan     0.2306   -0.1182
##   1900       26.5242             nan     0.2306   -0.2009
##   1920       26.4862             nan     0.2306   -0.1157
##   1940       26.4280             nan     0.2306   -0.1249
##   1960       26.3105             nan     0.2306   -0.1185
##   1980       26.2337             nan     0.2306   -0.1599
##   2000       26.1559             nan     0.2306   -0.1213
##   2020       26.0984             nan     0.2306   -0.0990
##   2040       26.0258             nan     0.2306   -0.0645
##   2060       26.0332             nan     0.2306   -0.1599
##   2080       25.9343             nan     0.2306   -0.0734
##   2100       25.9254             nan     0.2306   -0.0652
##   2120       25.8465             nan     0.2306   -0.0960
##   2140       25.7692             nan     0.2306   -0.1480
##   2160       25.6890             nan     0.2306   -0.1098
##   2180       25.5939             nan     0.2306   -0.0725
##   2200       25.5302             nan     0.2306   -0.1502
##   2220       25.4763             nan     0.2306   -0.1986
##   2240       25.4023             nan     0.2306   -0.2125
##   2260       25.3471             nan     0.2306   -0.0712
##   2280       25.3249             nan     0.2306   -0.1573
##   2300       25.2497             nan     0.2306   -0.0913
##   2320       25.1908             nan     0.2306   -0.0971
##   2340       25.1392             nan     0.2306   -0.1003
##   2360       25.0913             nan     0.2306   -0.1520
##   2380       25.0346             nan     0.2306   -0.1765
##   2400       24.9952             nan     0.2306   -0.0902
##   2420       24.9476             nan     0.2306   -0.1496
##   2440       24.8756             nan     0.2306   -0.1015
##   2460       24.8935             nan     0.2306   -0.1575
##   2480       24.8325             nan     0.2306   -0.1369
##   2500       24.7295             nan     0.2306   -0.0833
##   2520       24.7552             nan     0.2306   -0.1138
##   2540       24.7335             nan     0.2306   -0.0892
##   2560       24.7007             nan     0.2306   -0.0727
##   2580       24.6701             nan     0.2306   -0.2264
##   2600       24.6286             nan     0.2306   -0.1302
##   2620       24.5795             nan     0.2306   -0.1299
##   2640       24.5283             nan     0.2306   -0.0933
##   2660       24.5215             nan     0.2306   -0.1407
##   2680       24.5017             nan     0.2306   -0.0754
##   2700       24.4857             nan     0.2306   -0.1643
##   2720       24.4955             nan     0.2306   -0.1050
##   2740       24.4636             nan     0.2306   -0.0847
##   2760       24.4040             nan     0.2306   -0.1042
##   2780       24.3654             nan     0.2306   -0.1121
##   2800       24.3338             nan     0.2306   -0.0876
##   2820       24.3491             nan     0.2306   -0.0877
##   2840       24.2922             nan     0.2306   -0.0985
##   2860       24.2357             nan     0.2306   -0.0984
##   2880       24.2008             nan     0.2306   -0.1560
##   2900       24.1793             nan     0.2306   -0.0503
##   2920       24.1016             nan     0.2306   -0.1304
##   2940       24.0673             nan     0.2306   -0.1723
##   2960       24.0149             nan     0.2306   -0.0551
##   2980       23.9990             nan     0.2306   -0.0836
##   3000       23.9951             nan     0.2306   -0.0907
##   3020       23.9694             nan     0.2306   -0.1882
##   3040       23.9402             nan     0.2306   -0.1880
##   3060       23.8905             nan     0.2306   -0.2435
##   3080       23.8815             nan     0.2306   -0.1768
##   3100       23.8178             nan     0.2306   -0.1400
##   3120       23.8653             nan     0.2306   -0.0735
##   3140       23.7836             nan     0.2306   -0.1773
##   3160       23.7492             nan     0.2306   -0.1066
##   3180       23.7274             nan     0.2306   -0.1466
##   3200       23.6721             nan     0.2306   -0.0547
##   3220       23.6558             nan     0.2306   -0.1184
##   3240       23.6412             nan     0.2306   -0.1136
##   3260       23.6023             nan     0.2306   -0.1368
##   3280       23.6091             nan     0.2306   -0.1070
##   3300       23.5638             nan     0.2306   -0.1044
##   3320       23.5612             nan     0.2306   -0.1122
##   3340       23.5538             nan     0.2306   -0.1314
##   3360       23.5532             nan     0.2306   -0.0703
##   3380       23.5061             nan     0.2306   -0.2079
##   3400       23.4527             nan     0.2306   -0.0993
##   3420       23.4089             nan     0.2306   -0.1353
##   3440       23.3772             nan     0.2306   -0.1417
##   3460       23.4300             nan     0.2306   -0.1320
##   3480       23.4011             nan     0.2306   -0.1086
##   3500       23.3875             nan     0.2306   -0.0988
##   3520       23.3403             nan     0.2306   -0.1481
##   3540       23.3376             nan     0.2306   -0.0600
##   3560       23.3049             nan     0.2306   -0.1296
##   3580       23.2585             nan     0.2306   -0.1017
##   3600       23.2468             nan     0.2306   -0.0890
##   3620       23.2244             nan     0.2306   -0.0580
##   3640       23.1967             nan     0.2306   -0.1625
##   3660       23.1718             nan     0.2306   -0.1127
##   3680       23.1465             nan     0.2306   -0.0660
##   3700       23.1517             nan     0.2306   -0.0973
##   3720       23.1168             nan     0.2306   -0.0579
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      672.2928             nan     0.3896  502.1240
##      2      468.8291             nan     0.3896  202.9958
##      3      373.2304             nan     0.3896   91.2630
##      4      325.7640             nan     0.3896   45.9424
##      5      300.9983             nan     0.3896   23.7617
##      6      285.5462             nan     0.3896   14.4377
##      7      270.9259             nan     0.3896   14.1772
##      8      262.2058             nan     0.3896    7.7425
##      9      254.9525             nan     0.3896    6.0039
##     10      244.5546             nan     0.3896   10.4672
##     20      204.2044             nan     0.3896    0.4214
##     40      171.9674             nan     0.3896    0.5796
##     60      155.5397             nan     0.3896    0.0694
##     80      141.6836             nan     0.3896   -0.0275
##    100      132.1458             nan     0.3896   -0.1046
##    120      124.7488             nan     0.3896   -0.0552
##    140      120.0868             nan     0.3896   -0.1835
##    160      115.1790             nan     0.3896    0.0835
##    180      110.1663             nan     0.3896   -0.1350
##    200      106.6357             nan     0.3896    0.1294
##    220      103.2750             nan     0.3896   -0.1403
##    240      101.0129             nan     0.3896   -0.3550
##    260       98.7296             nan     0.3896   -0.2488
##    280       95.5102             nan     0.3896   -0.0320
##    300       93.0493             nan     0.3896   -0.0371
##    320       90.9065             nan     0.3896   -0.0818
##    340       88.9585             nan     0.3896   -0.1094
##    360       86.9967             nan     0.3896    0.0113
##    380       85.0310             nan     0.3896   -0.1480
##    400       83.3119             nan     0.3896   -0.3268
##    420       82.0764             nan     0.3896   -0.0950
##    440       80.5235             nan     0.3896   -0.1702
##    460       79.0738             nan     0.3896   -0.0765
##    480       77.8660             nan     0.3896   -0.0070
##    500       76.7659             nan     0.3896   -0.1939
##    520       75.4761             nan     0.3896   -0.1132
##    540       74.2228             nan     0.3896   -0.2946
##    560       73.1614             nan     0.3896   -0.0490
##    580       72.3256             nan     0.3896   -0.0422
##    600       71.3000             nan     0.3896   -0.1747
##    620       70.4333             nan     0.3896   -0.2136
##    640       69.2329             nan     0.3896   -0.0623
##    660       68.2587             nan     0.3896   -0.0844
##    680       67.2746             nan     0.3896   -0.2405
##    700       66.4487             nan     0.3896   -0.1245
##    720       65.4298             nan     0.3896   -0.1606
##    740       64.6610             nan     0.3896   -0.0281
##    760       63.7985             nan     0.3896   -0.1438
##    780       62.9984             nan     0.3896   -0.1443
##    800       62.3878             nan     0.3896   -0.1029
##    820       61.6125             nan     0.3896   -0.0702
##    840       60.7782             nan     0.3896   -0.1082
##    860       60.1579             nan     0.3896   -0.1305
##    880       59.5151             nan     0.3896   -0.1190
##    900       58.9498             nan     0.3896   -0.1683
##    920       58.4945             nan     0.3896   -0.1848
##    940       57.9744             nan     0.3896   -0.1616
##    960       57.5913             nan     0.3896   -0.1971
##    980       57.0559             nan     0.3896   -0.0654
##   1000       56.5428             nan     0.3896   -0.0835
##   1020       56.0524             nan     0.3896   -0.1548
##   1040       55.7051             nan     0.3896   -0.1439
##   1060       55.2176             nan     0.3896   -0.0929
##   1080       54.8356             nan     0.3896   -0.0817
##   1100       54.5203             nan     0.3896   -0.1427
##   1120       54.1719             nan     0.3896   -0.0524
##   1140       53.6079             nan     0.3896   -0.1075
##   1160       53.2811             nan     0.3896   -0.0482
##   1180       52.9364             nan     0.3896   -0.0517
##   1200       52.5529             nan     0.3896   -0.0878
##   1220       52.4031             nan     0.3896   -0.1083
##   1240       52.1374             nan     0.3896   -0.1739
##   1260       51.7621             nan     0.3896   -0.1317
##   1280       51.4563             nan     0.3896   -0.1591
##   1300       51.1539             nan     0.3896   -0.1190
##   1320       50.8621             nan     0.3896   -0.2975
##   1340       50.4855             nan     0.3896   -0.1398
##   1360       50.1061             nan     0.3896   -0.1451
##   1380       49.9010             nan     0.3896   -0.1168
##   1400       49.5773             nan     0.3896   -0.0791
##   1420       49.2619             nan     0.3896   -0.0283
##   1440       48.9560             nan     0.3896   -0.1042
##   1460       48.6519             nan     0.3896   -0.1079
##   1480       48.3948             nan     0.3896   -0.0216
##   1500       48.1869             nan     0.3896   -0.1422
##   1520       47.8576             nan     0.3896   -0.0621
##   1540       47.5032             nan     0.3896   -0.1407
##   1560       47.2837             nan     0.3896   -0.0434
##   1580       47.0235             nan     0.3896   -0.0578
##   1600       46.8355             nan     0.3896   -0.1391
##   1620       46.5850             nan     0.3896   -0.0107
##   1640       46.2851             nan     0.3896   -0.0727
##   1660       46.0476             nan     0.3896   -0.0829
##   1680       45.8815             nan     0.3896   -0.1056
##   1700       45.6395             nan     0.3896   -0.0751
##   1720       45.4020             nan     0.3896   -0.1121
##   1740       45.1187             nan     0.3896   -0.0631
##   1760       44.9604             nan     0.3896   -0.0813
##   1780       44.8103             nan     0.3896   -0.1420
##   1800       44.7552             nan     0.3896   -0.1089
##   1820       44.5283             nan     0.3896   -0.1610
##   1840       44.3517             nan     0.3896   -0.0852
##   1860       44.0921             nan     0.3896   -0.1286
##   1880       43.8645             nan     0.3896   -0.0635
##   1900       43.6592             nan     0.3896   -0.0743
##   1920       43.4406             nan     0.3896   -0.1241
##   1940       43.3249             nan     0.3896   -0.1290
##   1960       43.1072             nan     0.3896   -0.0964
##   1980       42.9389             nan     0.3896   -0.1754
##   2000       42.7217             nan     0.3896   -0.0554
##   2020       42.5560             nan     0.3896   -0.1556
##   2040       42.3197             nan     0.3896   -0.1205
##   2060       42.1095             nan     0.3896   -0.0815
##   2080       42.1211             nan     0.3896   -0.4315
##   2100       41.8606             nan     0.3896   -0.1479
##   2120       41.6666             nan     0.3896   -0.0858
##   2140       41.5360             nan     0.3896   -0.1198
##   2160       41.4672             nan     0.3896   -0.1682
##   2180       41.2682             nan     0.3896   -0.1166
##   2200       41.0692             nan     0.3896   -0.0673
##   2220       40.8680             nan     0.3896   -0.0759
##   2240       40.6845             nan     0.3896   -0.1000
##   2260       40.5514             nan     0.3896   -0.0721
##   2280       40.4866             nan     0.3896   -0.0714
##   2300       40.3256             nan     0.3896   -0.0852
##   2320       40.1994             nan     0.3896   -0.0645
##   2340       40.0432             nan     0.3896   -0.0780
##   2360       39.8786             nan     0.3896   -0.0376
##   2380       39.7998             nan     0.3896   -0.1558
##   2400       39.6718             nan     0.3896   -0.2468
##   2420       39.5642             nan     0.3896   -0.0125
##   2440       39.4364             nan     0.3896   -0.1086
##   2460       39.3158             nan     0.3896   -0.0223
##   2480       39.2725             nan     0.3896   -0.1183
##   2500       39.0910             nan     0.3896   -0.0667
##   2520       38.9101             nan     0.3896   -0.0691
##   2540       38.7957             nan     0.3896   -0.0745
##   2560       38.6868             nan     0.3896   -0.0642
##   2580       38.4327             nan     0.3896   -0.1084
##   2600       38.3856             nan     0.3896   -0.1249
##   2620       38.3827             nan     0.3896   -0.1012
##   2640       38.1964             nan     0.3896   -0.0952
##   2660       38.1900             nan     0.3896   -0.1800
##   2680       37.9421             nan     0.3896   -0.0780
##   2700       37.8584             nan     0.3896   -0.1163
##   2720       37.7215             nan     0.3896   -0.1507
##   2740       37.5946             nan     0.3896   -0.0467
##   2760       37.5014             nan     0.3896   -0.0576
##   2780       37.4265             nan     0.3896   -0.0753
##   2800       37.2722             nan     0.3896   -0.0368
##   2820       37.1668             nan     0.3896   -0.0403
##   2840       37.1104             nan     0.3896   -0.1121
##   2860       37.0961             nan     0.3896   -0.0977
##   2880       37.0555             nan     0.3896   -0.0575
##   2900       36.8989             nan     0.3896   -0.1192
##   2920       36.7971             nan     0.3896   -0.1223
##   2940       36.7313             nan     0.3896   -0.0757
##   2960       36.5919             nan     0.3896   -0.1070
##   2980       36.5567             nan     0.3896   -0.0918
##   3000       36.4663             nan     0.3896   -0.0495
##   3020       36.3946             nan     0.3896   -0.0797
##   3040       36.3119             nan     0.3896   -0.0674
##   3060       36.2611             nan     0.3896   -0.1382
##   3080       36.1556             nan     0.3896   -0.0164
##   3100       36.0812             nan     0.3896   -0.1237
##   3120       35.9914             nan     0.3896   -0.0693
##   3140       35.9102             nan     0.3896   -0.0753
##   3160       35.8598             nan     0.3896   -0.1289
##   3180       35.7090             nan     0.3896   -0.0674
##   3200       35.6170             nan     0.3896   -0.0515
##   3220       35.5703             nan     0.3896   -0.0703
##   3240       35.4779             nan     0.3896   -0.1079
##   3260       35.4500             nan     0.3896   -0.1812
##   3280       35.4186             nan     0.3896   -0.1514
##   3300       35.3465             nan     0.3896   -0.0478
##   3320       35.2811             nan     0.3896   -0.1351
##   3340       35.2310             nan     0.3896   -0.1079
##   3360       35.1391             nan     0.3896   -0.0602
##   3380       35.0956             nan     0.3896   -0.1036
##   3400       35.0512             nan     0.3896   -0.1380
##   3420       34.9733             nan     0.3896   -0.1474
##   3440       34.8654             nan     0.3896   -0.0503
##   3460       34.7549             nan     0.3896   -0.0301
##   3480       34.6856             nan     0.3896   -0.0473
##   3500       34.6568             nan     0.3896   -0.0720
##   3520       34.6509             nan     0.3896   -0.0805
##   3540       34.5544             nan     0.3896   -0.0728
##   3560       34.4874             nan     0.3896   -0.0901
##   3580       34.4388             nan     0.3896   -0.0789
##   3600       34.3650             nan     0.3896   -0.0359
##   3620       34.2992             nan     0.3896   -0.0524
##   3640       34.2083             nan     0.3896   -0.1112
##   3660       34.1309             nan     0.3896   -0.0811
##   3680       34.0759             nan     0.3896   -0.0659
##   3700       33.9991             nan     0.3896   -0.0599
##   3720       34.0195             nan     0.3896   -0.1157
##   3740       33.9021             nan     0.3896   -0.1010
##   3760       33.8605             nan     0.3896   -0.0676
##   3780       33.8399             nan     0.3896   -0.3372
##   3800       33.6721             nan     0.3896   -0.0778
##   3820       33.6651             nan     0.3896   -0.0773
##   3840       33.5578             nan     0.3896   -0.0943
##   3860       33.5280             nan     0.3896   -0.0809
##   3880       33.4816             nan     0.3896   -0.0736
##   3900       33.4687             nan     0.3896   -0.0832
##   3920       33.3714             nan     0.3896   -0.1173
##   3940       33.3102             nan     0.3896   -0.1358
##   3960       33.2368             nan     0.3896   -0.1130
##   3980       33.2219             nan     0.3896   -0.1469
##   4000       33.1143             nan     0.3896   -0.0863
##   4020       33.1301             nan     0.3896   -0.1614
##   4040       33.0614             nan     0.3896   -0.0709
##   4060       32.9756             nan     0.3896   -0.0843
##   4080       32.9398             nan     0.3896   -0.0840
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      546.2948             nan     0.5470  646.6852
##      2      378.2894             nan     0.5470  163.8886
##      3      321.7950             nan     0.5470   52.8618
##      4      293.7752             nan     0.5470   25.7071
##      5      278.4390             nan     0.5470   15.5100
##      6      266.7310             nan     0.5470   11.0675
##      7      255.6124             nan     0.5470    9.3796
##      8      245.9038             nan     0.5470    9.1618
##      9      239.9977             nan     0.5470    4.6418
##     10      235.6126             nan     0.5470    3.2009
##     20      199.2822             nan     0.5470    1.2231
##     40      163.1671             nan     0.5470   -0.0040
##     60      148.7943             nan     0.5470   -0.2149
##     80      138.7075             nan     0.5470   -0.4525
##    100      129.6375             nan     0.5470   -0.0188
##    120      122.2797             nan     0.5470   -0.3828
##    140      115.4769             nan     0.5470   -0.0544
##    160      109.6355             nan     0.5470   -0.3433
##    180      105.3269             nan     0.5470   -0.3373
##    200      102.2365             nan     0.5470   -0.3365
##    220       99.4703             nan     0.5470   -0.1588
##    240       96.4003             nan     0.5470   -0.2000
##    260       94.0561             nan     0.5470   -0.4103
##    280       91.2289             nan     0.5470   -0.3090
##    300       88.4029             nan     0.5470   -0.2524
##    320       86.3056             nan     0.5470   -0.1320
##    340       83.9285             nan     0.5470   -0.4618
##    360       82.3247             nan     0.5470   -0.2013
##    380       81.1313             nan     0.5470   -0.1909
##    400       78.9120             nan     0.5470   -0.2479
##    420       77.2574             nan     0.5470   -0.1592
##    440       75.6182             nan     0.5470   -0.1872
##    460       74.3703             nan     0.5470   -0.3239
##    480       73.1834             nan     0.5470   -0.2688
##    500       72.1455             nan     0.5470   -0.2169
##    520       71.0772             nan     0.5470   -0.3412
##    540       70.2103             nan     0.5470   -0.1895
##    560       69.3643             nan     0.5470   -0.1231
##    580       68.2779             nan     0.5470   -0.2247
##    600       67.4116             nan     0.5470   -0.1835
##    620       66.7119             nan     0.5470   -0.0942
##    640       65.8136             nan     0.5470   -0.1245
##    660       64.9947             nan     0.5470   -0.2303
##    680       64.1698             nan     0.5470   -0.2024
##    700       63.3151             nan     0.5470   -0.2006
##    720       62.7323             nan     0.5470   -0.1402
##    740       62.0844             nan     0.5470   -0.1707
##    760       61.6574             nan     0.5470   -0.1431
##    780       60.9029             nan     0.5470   -0.0909
##    800       60.3089             nan     0.5470   -0.2176
##    820       59.4568             nan     0.5470    0.0028
##    840       58.6875             nan     0.5470   -0.1739
##    860       58.3116             nan     0.5470   -0.1101
##    880       57.5041             nan     0.5470   -0.1423
##    900       57.1253             nan     0.5470   -0.1580
##    920       56.5548             nan     0.5470   -0.1564
##    940       56.2386             nan     0.5470   -0.2200
##    960       55.6240             nan     0.5470   -0.1016
##    980       55.0811             nan     0.5470   -0.0577
##   1000       54.6083             nan     0.5470   -0.1289
##   1020       54.2450             nan     0.5470   -0.1850
##   1040       53.9240             nan     0.5470   -0.2109
##   1060       53.4811             nan     0.5470   -0.0486
##   1080       53.2204             nan     0.5470   -0.1069
##   1100       52.8317             nan     0.5470   -0.2187
##   1120       52.3022             nan     0.5470   -0.1560
##   1140       52.0151             nan     0.5470   -0.3351
##   1160       51.5292             nan     0.5470   -0.2224
##   1180       51.1206             nan     0.5470   -0.1138
##   1200       50.7515             nan     0.5470   -0.0862
##   1220       50.4533             nan     0.5470   -0.0963
##   1240       50.2646             nan     0.5470   -0.1898
##   1260       49.9070             nan     0.5470   -0.2007
##   1280       49.4419             nan     0.5470   -0.1435
##   1300       49.0973             nan     0.5470   -0.1621
##   1320       48.7511             nan     0.5470   -0.2605
##   1340       48.4590             nan     0.5470   -0.1636
##   1360       48.0815             nan     0.5470   -0.1647
##   1380       47.9041             nan     0.5470   -0.3045
##   1400       47.8782             nan     0.5470   -0.3376
##   1420       47.5136             nan     0.5470   -0.3805
##   1440       47.2475             nan     0.5470   -0.2492
##   1460       47.0586             nan     0.5470   -0.4874
##   1480       46.9318             nan     0.5470   -0.2200
##   1500       46.7240             nan     0.5470   -0.0487
##   1520       46.4873             nan     0.5470   -0.2823
##   1540       46.3077             nan     0.5470   -0.2648
##   1560       46.0930             nan     0.5470   -0.0698
##   1580       45.8568             nan     0.5470   -0.1801
##   1600       45.5679             nan     0.5470   -0.2215
##   1620       45.3332             nan     0.5470   -0.0180
##   1640       45.2090             nan     0.5470   -0.0639
##   1660       45.1088             nan     0.5470   -0.2155
##   1680       44.8143             nan     0.5470   -0.1830
##   1700       44.6326             nan     0.5470   -0.3710
##   1720       44.5032             nan     0.5470   -0.2767
##   1740       44.2053             nan     0.5470   -0.1053
##   1760       44.1570             nan     0.5470   -0.2717
##   1780       43.9184             nan     0.5470   -0.1711
##   1800       43.6966             nan     0.5470   -0.1875
##   1820       43.5577             nan     0.5470   -0.1331
##   1840       43.3955             nan     0.5470   -0.1207
##   1860       43.2338             nan     0.5470   -0.1692
##   1880       42.9989             nan     0.5470   -0.1791
##   1900       42.8895             nan     0.5470   -0.1618
##   1920       42.7325             nan     0.5470   -0.1135
##   1940       42.5153             nan     0.5470   -0.0952
##   1960       42.4507             nan     0.5470   -0.0847
##   1980       42.4300             nan     0.5470   -0.1617
##   2000       42.2852             nan     0.5470   -0.1925
##   2020       42.1706             nan     0.5470   -0.1224
##   2040       42.0260             nan     0.5470   -0.0835
##   2060       41.8698             nan     0.5470   -0.1809
##   2080       41.7229             nan     0.5470   -0.1633
##   2100       41.3905             nan     0.5470   -0.2854
##   2120       41.3040             nan     0.5470   -0.1320
##   2140       41.1448             nan     0.5470   -0.1122
##   2160       41.0695             nan     0.5470   -0.1080
##   2180       40.8824             nan     0.5470   -0.1621
##   2200       40.6822             nan     0.5470   -0.0863
##   2220       40.6808             nan     0.5470   -0.1476
##   2240       40.5806             nan     0.5470   -0.1180
##   2260       40.3749             nan     0.5470   -0.1889
##   2280       40.3416             nan     0.5470   -0.2041
##   2300       40.3199             nan     0.5470   -0.2025
##   2320       40.0482             nan     0.5470   -0.1703
##   2340       39.8630             nan     0.5470   -0.1089
##   2360       39.7439             nan     0.5470   -0.2013
##   2380       39.7414             nan     0.5470   -0.3464
##   2400       39.5554             nan     0.5470   -0.0857
##   2420       39.4500             nan     0.5470   -0.0765
##   2440       39.3318             nan     0.5470   -0.0608
##   2460       39.2688             nan     0.5470   -0.2066
##   2480       39.1404             nan     0.5470   -0.1837
##   2500       39.0650             nan     0.5470   -0.1807
##   2520       39.0264             nan     0.5470   -0.2322
##   2540       38.8263             nan     0.5470   -0.0932
##   2560       38.6978             nan     0.5470   -0.1369
##   2580       38.6525             nan     0.5470   -0.0984
##   2600       38.5024             nan     0.5470   -0.2570
##   2620       38.3330             nan     0.5470   -0.1617
##   2640       38.3354             nan     0.5470   -0.1706
##   2660       38.2057             nan     0.5470   -0.2021
##   2680       38.1418             nan     0.5470   -0.3236
##   2700       38.0967             nan     0.5470   -0.1170
##   2720       38.0542             nan     0.5470   -0.1910
##   2740       37.9633             nan     0.5470   -0.0686
##   2760       37.8519             nan     0.5470   -0.1214
##   2780       37.8009             nan     0.5470   -0.1629
##   2800       37.5930             nan     0.5470   -0.1508
##   2820       37.5071             nan     0.5470   -0.1648
##   2840       37.3790             nan     0.5470   -0.1243
##   2860       37.3053             nan     0.5470   -0.3473
##   2880       37.1731             nan     0.5470   -0.0932
##   2900       37.1060             nan     0.5470   -0.0827
##   2920       37.0209             nan     0.5470   -0.2038
##   2940       36.9293             nan     0.5470   -0.1604
##   2960       36.8635             nan     0.5470   -0.1156
##   2980       36.7335             nan     0.5470   -0.1661
##   3000       36.6274             nan     0.5470   -0.1797
##   3020       36.5727             nan     0.5470   -0.1609
##   3040       36.4889             nan     0.5470   -0.3193
##   3060       36.3668             nan     0.5470   -0.1462
##   3080       36.3386             nan     0.5470   -0.1691
##   3100       36.2291             nan     0.5470   -0.1268
##   3120       36.1105             nan     0.5470   -0.0544
##   3140       36.0828             nan     0.5470   -0.0783
##   3160       36.0685             nan     0.5470   -0.0893
##   3180       35.9234             nan     0.5470   -0.2233
##   3200       35.8534             nan     0.5470   -0.0961
##   3220       35.7648             nan     0.5470   -0.1794
##   3240       35.7871             nan     0.5470   -0.0977
##   3260       35.6705             nan     0.5470   -0.1913
##   3280       35.6304             nan     0.5470   -0.2171
##   3300       35.5863             nan     0.5470   -0.0343
##   3320       35.5213             nan     0.5470   -0.3126
##   3340       35.3469             nan     0.5470   -0.0598
##   3360       35.2604             nan     0.5470   -0.0945
##   3380       35.2411             nan     0.5470   -0.0335
##   3400       35.1440             nan     0.5470   -0.1289
##   3420       35.1051             nan     0.5470   -0.1339
##   3440       34.9862             nan     0.5470   -0.1835
##   3460       34.9795             nan     0.5470   -0.1039
##   3480       34.9225             nan     0.5470   -0.2075
##   3489       34.8666             nan     0.5470   -0.0450
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      820.8455             nan     0.2306  354.9822
##      2      601.3447             nan     0.2306  221.3719
##      3      462.4436             nan     0.2306  142.1719
##      4      372.0282             nan     0.2306   88.0553
##      5      316.0236             nan     0.2306   54.6381
##      6      276.6129             nan     0.2306   38.8663
##      7      249.8146             nan     0.2306   25.2368
##      8      230.2540             nan     0.2306   18.0238
##      9      215.7105             nan     0.2306   13.2429
##     10      205.2733             nan     0.2306   10.4246
##     20      162.6827             nan     0.2306    1.6512
##     40      130.8305             nan     0.2306    0.1719
##     60      115.2979             nan     0.2306    0.2337
##     80      105.0743             nan     0.2306    0.0607
##    100       95.6104             nan     0.2306    0.0922
##    120       88.4424             nan     0.2306    0.0678
##    140       82.6230             nan     0.2306    0.0993
##    160       77.4984             nan     0.2306   -0.2168
##    180       73.1477             nan     0.2306    0.0106
##    200       69.3653             nan     0.2306   -0.2684
##    220       66.1511             nan     0.2306   -0.0490
##    240       63.4546             nan     0.2306   -0.1301
##    260       61.0822             nan     0.2306   -0.1345
##    280       58.8076             nan     0.2306   -0.0783
##    300       56.8362             nan     0.2306   -0.1391
##    320       55.2592             nan     0.2306   -0.2477
##    340       53.6932             nan     0.2306   -0.1620
##    360       52.1652             nan     0.2306   -0.2156
##    380       50.6490             nan     0.2306   -0.1580
##    400       49.2110             nan     0.2306   -0.2440
##    420       47.9137             nan     0.2306   -0.2674
##    440       46.6894             nan     0.2306   -0.1133
##    460       45.6235             nan     0.2306   -0.1626
##    480       44.6172             nan     0.2306   -0.0637
##    500       43.6384             nan     0.2306   -0.1232
##    520       42.9310             nan     0.2306   -0.1872
##    540       42.0784             nan     0.2306   -0.2099
##    560       41.3450             nan     0.2306   -0.1478
##    580       40.5073             nan     0.2306   -0.2028
##    600       39.7866             nan     0.2306   -0.1737
##    620       39.1431             nan     0.2306   -0.1512
##    640       38.3727             nan     0.2306   -0.0302
##    660       37.8539             nan     0.2306   -0.0958
##    680       37.3364             nan     0.2306   -0.0593
##    700       36.7116             nan     0.2306   -0.1116
##    720       36.1723             nan     0.2306   -0.1101
##    740       35.5622             nan     0.2306   -0.0697
##    760       35.2239             nan     0.2306   -0.1248
##    780       34.7138             nan     0.2306   -0.0543
##    800       34.3137             nan     0.2306   -0.0901
##    820       33.9592             nan     0.2306   -0.1319
##    840       33.5209             nan     0.2306   -0.1007
##    860       33.2034             nan     0.2306   -0.1304
##    880       32.9071             nan     0.2306   -0.1025
##    900       32.5408             nan     0.2306   -0.1662
##    920       32.2455             nan     0.2306   -0.1753
##    940       31.9424             nan     0.2306   -0.1323
##    960       31.6492             nan     0.2306   -0.0516
##    980       31.3611             nan     0.2306   -0.2308
##   1000       31.1184             nan     0.2306   -0.1357
##   1020       30.9322             nan     0.2306   -0.1751
##   1040       30.6770             nan     0.2306   -0.1206
##   1060       30.3561             nan     0.2306   -0.1079
##   1080       30.2245             nan     0.2306   -0.0943
##   1100       29.9961             nan     0.2306   -0.1016
##   1120       29.8451             nan     0.2306   -0.1242
##   1140       29.6636             nan     0.2306   -0.2337
##   1160       29.4589             nan     0.2306   -0.1522
##   1180       29.2674             nan     0.2306   -0.0378
##   1200       29.1133             nan     0.2306   -0.0541
##   1220       28.9750             nan     0.2306   -0.0804
##   1240       28.7649             nan     0.2306   -0.0546
##   1260       28.5861             nan     0.2306   -0.1158
##   1280       28.4401             nan     0.2306   -0.1958
##   1300       28.2575             nan     0.2306   -0.0755
##   1320       28.1019             nan     0.2306   -0.1030
##   1340       28.0197             nan     0.2306   -0.0727
##   1360       27.8332             nan     0.2306   -0.0924
##   1380       27.6418             nan     0.2306   -0.1491
##   1400       27.5299             nan     0.2306   -0.2789
##   1420       27.3899             nan     0.2306   -0.1038
##   1440       27.2938             nan     0.2306   -0.1312
##   1460       27.1269             nan     0.2306   -0.0834
##   1480       27.0086             nan     0.2306   -0.1189
##   1500       26.9040             nan     0.2306   -0.1318
##   1520       26.7718             nan     0.2306   -0.1507
##   1540       26.6671             nan     0.2306   -0.1589
##   1560       26.6276             nan     0.2306   -0.1045
##   1580       26.5524             nan     0.2306   -0.0850
##   1600       26.4420             nan     0.2306   -0.0789
##   1620       26.3800             nan     0.2306   -0.0967
##   1640       26.3203             nan     0.2306   -0.0728
##   1660       26.2180             nan     0.2306   -0.1067
##   1680       26.1285             nan     0.2306   -0.0919
##   1700       26.0193             nan     0.2306   -0.0876
##   1720       25.9382             nan     0.2306   -0.2086
##   1740       25.9106             nan     0.2306   -0.0943
##   1760       25.8118             nan     0.2306   -0.2383
##   1780       25.7735             nan     0.2306   -0.1156
##   1800       25.6633             nan     0.2306   -0.0781
##   1820       25.6001             nan     0.2306   -0.1508
##   1840       25.5471             nan     0.2306   -0.2581
##   1860       25.5071             nan     0.2306   -0.1536
##   1880       25.4010             nan     0.2306   -0.1116
##   1900       25.3121             nan     0.2306   -0.1107
##   1920       25.2249             nan     0.2306   -0.1343
##   1940       25.1992             nan     0.2306   -0.1216
##   1960       25.1093             nan     0.2306   -0.1900
##   1980       25.0284             nan     0.2306   -0.1138
##   2000       24.9462             nan     0.2306   -0.1025
##   2020       24.8520             nan     0.2306   -0.0827
##   2040       24.8091             nan     0.2306   -0.1176
##   2060       24.7543             nan     0.2306   -0.1729
##   2080       24.6769             nan     0.2306   -0.1087
##   2100       24.5993             nan     0.2306   -0.1143
##   2120       24.5210             nan     0.2306   -0.0930
##   2140       24.5088             nan     0.2306   -0.1709
##   2160       24.4415             nan     0.2306   -0.0719
##   2180       24.4091             nan     0.2306   -0.1482
##   2200       24.3308             nan     0.2306   -0.0760
##   2220       24.3125             nan     0.2306   -0.0832
##   2240       24.2587             nan     0.2306   -0.1730
##   2260       24.2549             nan     0.2306   -0.1995
##   2280       24.2311             nan     0.2306   -0.4088
##   2300       24.1390             nan     0.2306   -0.1089
##   2320       24.1055             nan     0.2306   -0.0811
##   2340       24.0185             nan     0.2306   -0.0769
##   2360       23.9985             nan     0.2306   -0.0715
##   2380       23.9474             nan     0.2306   -0.0760
##   2400       23.8886             nan     0.2306   -0.1033
##   2420       23.8287             nan     0.2306   -0.0929
##   2440       23.8317             nan     0.2306   -0.0867
##   2460       23.7180             nan     0.2306   -0.0782
##   2480       23.7171             nan     0.2306   -0.1096
##   2500       23.7327             nan     0.2306   -0.1386
##   2520       23.6725             nan     0.2306   -0.0839
##   2540       23.6641             nan     0.2306   -0.1258
##   2560       23.6380             nan     0.2306   -0.1406
##   2580       23.5763             nan     0.2306   -0.0818
##   2600       23.4928             nan     0.2306   -0.1596
##   2620       23.4754             nan     0.2306   -0.1141
##   2640       23.4244             nan     0.2306   -0.1122
##   2660       23.4068             nan     0.2306   -0.0707
##   2680       23.3700             nan     0.2306   -0.1072
##   2700       23.3937             nan     0.2306   -0.0594
##   2720       23.3372             nan     0.2306   -0.0901
##   2740       23.2851             nan     0.2306   -0.0870
##   2760       23.2509             nan     0.2306   -0.1403
##   2780       23.2248             nan     0.2306   -0.1162
##   2800       23.2223             nan     0.2306   -0.0830
##   2820       23.1378             nan     0.2306   -0.1100
##   2840       23.1477             nan     0.2306   -0.0739
##   2860       23.1184             nan     0.2306   -0.0443
##   2880       23.0516             nan     0.2306   -0.0942
##   2900       23.0343             nan     0.2306   -0.1441
##   2920       22.9955             nan     0.2306   -0.0795
##   2940       22.9627             nan     0.2306   -0.0969
##   2960       22.9186             nan     0.2306   -0.1463
##   2980       22.9171             nan     0.2306   -0.0503
##   3000       22.8413             nan     0.2306   -0.1440
##   3020       22.8432             nan     0.2306   -0.0830
##   3040       22.8031             nan     0.2306   -0.0561
##   3060       22.7793             nan     0.2306   -0.1274
##   3080       22.7646             nan     0.2306   -0.0808
##   3100       22.7344             nan     0.2306   -0.1910
##   3120       22.7732             nan     0.2306   -0.1088
##   3140       22.7272             nan     0.2306   -0.1098
##   3160       22.6877             nan     0.2306   -0.0581
##   3180       22.6992             nan     0.2306   -0.0559
##   3200       22.6681             nan     0.2306   -0.0425
##   3220       22.6572             nan     0.2306   -0.0952
##   3240       22.6080             nan     0.2306   -0.1073
##   3260       22.5732             nan     0.2306   -0.1589
##   3280       22.5537             nan     0.2306   -0.0945
##   3300       22.5530             nan     0.2306   -0.0704
##   3320       22.5272             nan     0.2306   -0.1838
##   3340       22.5175             nan     0.2306   -0.1509
##   3360       22.5171             nan     0.2306   -0.1300
##   3380       22.5199             nan     0.2306   -0.1240
##   3400       22.4720             nan     0.2306   -0.1304
##   3420       22.4254             nan     0.2306   -0.0897
##   3440       22.3961             nan     0.2306   -0.0845
##   3460       22.3754             nan     0.2306   -0.1086
##   3480       22.3219             nan     0.2306   -0.1136
##   3500       22.3381             nan     0.2306   -0.1780
##   3520       22.3273             nan     0.2306   -0.1204
##   3540       22.3090             nan     0.2306   -0.1400
##   3560       22.2884             nan     0.2306   -0.0890
##   3580       22.2583             nan     0.2306   -0.1676
##   3600       22.2299             nan     0.2306   -0.0843
##   3620       22.2379             nan     0.2306   -0.2166
##   3640       22.2244             nan     0.2306   -0.1199
##   3660       22.1460             nan     0.2306   -0.0753
##   3680       22.1563             nan     0.2306   -0.0849
##   3700       22.1860             nan     0.2306   -0.1065
##   3720       22.1490             nan     0.2306   -0.1010
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      676.0789             nan     0.3896  499.9389
##      2      457.5824             nan     0.3896  209.9930
##      3      367.0105             nan     0.3896   89.8997
##      4      317.9868             nan     0.3896   50.2982
##      5      288.4089             nan     0.3896   28.2877
##      6      272.0808             nan     0.3896   16.1806
##      7      260.7543             nan     0.3896    9.5682
##      8      252.5014             nan     0.3896    8.3156
##      9      244.5863             nan     0.3896    7.3241
##     10      239.1197             nan     0.3896    4.5480
##     20      199.4454             nan     0.3896    1.7038
##     40      168.6189             nan     0.3896    0.7368
##     60      150.9381             nan     0.3896   -0.2519
##     80      139.7138             nan     0.3896   -0.2871
##    100      131.0291             nan     0.3896   -0.0364
##    120      124.1422             nan     0.3896    0.2185
##    140      119.0259             nan     0.3896   -0.1551
##    160      114.4388             nan     0.3896    0.0591
##    180      110.4909             nan     0.3896   -0.0368
##    200      105.7577             nan     0.3896   -0.0705
##    220      102.5183             nan     0.3896    0.0297
##    240       99.0502             nan     0.3896    0.0340
##    260       96.6216             nan     0.3896   -0.1598
##    280       94.0879             nan     0.3896   -0.0930
##    300       91.5863             nan     0.3896   -0.0534
##    320       89.3238             nan     0.3896   -0.1809
##    340       86.9851             nan     0.3896   -0.1171
##    360       85.1070             nan     0.3896   -0.2688
##    380       83.1372             nan     0.3896   -0.0468
##    400       81.5594             nan     0.3896   -0.1133
##    420       80.0558             nan     0.3896   -0.1269
##    440       78.7844             nan     0.3896   -0.1188
##    460       77.5261             nan     0.3896   -0.1940
##    480       76.2224             nan     0.3896   -0.1700
##    500       74.7569             nan     0.3896   -0.1670
##    520       73.2339             nan     0.3896   -0.1225
##    540       71.9614             nan     0.3896   -0.1706
##    560       70.6810             nan     0.3896   -0.1161
##    580       69.6940             nan     0.3896   -0.2345
##    600       68.5283             nan     0.3896   -0.0489
##    620       67.5954             nan     0.3896   -0.1447
##    640       66.5349             nan     0.3896   -0.1046
##    660       65.6624             nan     0.3896   -0.2035
##    680       64.8135             nan     0.3896   -0.1296
##    700       64.0356             nan     0.3896   -0.1768
##    720       63.1804             nan     0.3896   -0.1331
##    740       62.2803             nan     0.3896   -0.1011
##    760       61.5865             nan     0.3896   -0.0284
##    780       60.9275             nan     0.3896   -0.1362
##    800       60.3281             nan     0.3896   -0.1209
##    820       59.7245             nan     0.3896   -0.0888
##    840       59.1819             nan     0.3896   -0.0956
##    860       58.5279             nan     0.3896   -0.0733
##    880       57.9908             nan     0.3896   -0.0488
##    900       57.5268             nan     0.3896   -0.1082
##    920       56.9183             nan     0.3896   -0.1119
##    940       56.3394             nan     0.3896   -0.1014
##    960       55.9408             nan     0.3896   -0.0925
##    980       55.3586             nan     0.3896   -0.1078
##   1000       54.6689             nan     0.3896   -0.1045
##   1020       54.3224             nan     0.3896   -0.1502
##   1040       53.8942             nan     0.3896   -0.0674
##   1060       53.4952             nan     0.3896   -0.1027
##   1080       53.0853             nan     0.3896   -0.1352
##   1100       52.5995             nan     0.3896   -0.1174
##   1120       52.1656             nan     0.3896   -0.1243
##   1140       51.7283             nan     0.3896   -0.0888
##   1160       51.3091             nan     0.3896   -0.1839
##   1180       50.8200             nan     0.3896   -0.1548
##   1200       50.4207             nan     0.3896   -0.1020
##   1220       50.0822             nan     0.3896   -0.0803
##   1240       49.7668             nan     0.3896   -0.1215
##   1260       49.3083             nan     0.3896   -0.0847
##   1280       49.0158             nan     0.3896   -0.0565
##   1300       48.6347             nan     0.3896   -0.0310
##   1320       48.1182             nan     0.3896   -0.1015
##   1340       47.8782             nan     0.3896   -0.1341
##   1360       47.6320             nan     0.3896   -0.2188
##   1380       47.3462             nan     0.3896   -0.0330
##   1400       47.0475             nan     0.3896   -0.2695
##   1420       46.7102             nan     0.3896   -0.0877
##   1440       46.2081             nan     0.3896   -0.0696
##   1460       45.8061             nan     0.3896   -0.0717
##   1480       45.4999             nan     0.3896   -0.0923
##   1500       45.2075             nan     0.3896   -0.1565
##   1520       44.9571             nan     0.3896   -0.0849
##   1540       44.8335             nan     0.3896   -0.1408
##   1560       44.5672             nan     0.3896   -0.2069
##   1580       44.3287             nan     0.3896   -0.0690
##   1600       44.0894             nan     0.3896   -0.1273
##   1620       43.8693             nan     0.3896   -0.1414
##   1640       43.6789             nan     0.3896   -0.1205
##   1660       43.3803             nan     0.3896   -0.1336
##   1680       43.1883             nan     0.3896   -0.0434
##   1700       43.0196             nan     0.3896   -0.0877
##   1720       42.6774             nan     0.3896   -0.0300
##   1740       42.5446             nan     0.3896   -0.1395
##   1760       42.3713             nan     0.3896   -0.0313
##   1780       42.1830             nan     0.3896   -0.0661
##   1800       41.9622             nan     0.3896   -0.0649
##   1820       41.8176             nan     0.3896   -0.0610
##   1840       41.6117             nan     0.3896   -0.0899
##   1860       41.3670             nan     0.3896   -0.1270
##   1880       41.2375             nan     0.3896   -0.2314
##   1900       41.0460             nan     0.3896   -0.0833
##   1920       40.8820             nan     0.3896   -0.1486
##   1940       40.6821             nan     0.3896   -0.0450
##   1960       40.4917             nan     0.3896   -0.1026
##   1980       40.2800             nan     0.3896   -0.0598
##   2000       40.0890             nan     0.3896   -0.0587
##   2020       39.8755             nan     0.3896   -0.1115
##   2040       39.7375             nan     0.3896   -0.1021
##   2060       39.6012             nan     0.3896   -0.0907
##   2080       39.5022             nan     0.3896   -0.0748
##   2100       39.3463             nan     0.3896   -0.0752
##   2120       39.2416             nan     0.3896   -0.1464
##   2140       39.0610             nan     0.3896   -0.1524
##   2160       38.8502             nan     0.3896   -0.0368
##   2180       38.6602             nan     0.3896   -0.0669
##   2200       38.4324             nan     0.3896   -0.0661
##   2220       38.2156             nan     0.3896   -0.0745
##   2240       38.0640             nan     0.3896   -0.1012
##   2260       37.9309             nan     0.3896   -0.0992
##   2280       37.7930             nan     0.3896   -0.0424
##   2300       37.6914             nan     0.3896   -0.0763
##   2320       37.4886             nan     0.3896   -0.0311
##   2340       37.3038             nan     0.3896   -0.0963
##   2360       37.1183             nan     0.3896   -0.0865
##   2380       37.0517             nan     0.3896   -0.1104
##   2400       36.9121             nan     0.3896   -0.0622
##   2420       36.8079             nan     0.3896   -0.0537
##   2440       36.6231             nan     0.3896   -0.0782
##   2460       36.4200             nan     0.3896   -0.0199
##   2480       36.3025             nan     0.3896    0.0098
##   2500       36.1643             nan     0.3896   -0.0845
##   2520       36.0804             nan     0.3896   -0.1302
##   2540       35.9442             nan     0.3896   -0.0949
##   2560       35.8479             nan     0.3896   -0.0959
##   2580       35.7413             nan     0.3896   -0.1038
##   2600       35.5764             nan     0.3896   -0.0774
##   2620       35.4393             nan     0.3896   -0.0918
##   2640       35.3948             nan     0.3896   -0.1244
##   2660       35.3002             nan     0.3896   -0.0834
##   2680       35.1201             nan     0.3896   -0.1994
##   2700       34.9479             nan     0.3896   -0.0880
##   2720       34.8885             nan     0.3896   -0.0662
##   2740       34.7747             nan     0.3896   -0.1117
##   2760       34.6756             nan     0.3896   -0.1346
##   2780       34.6000             nan     0.3896   -0.0417
##   2800       34.6503             nan     0.3896   -0.1389
##   2820       34.4205             nan     0.3896   -0.1587
##   2840       34.3233             nan     0.3896   -0.0770
##   2860       34.1981             nan     0.3896   -0.1535
##   2880       34.0937             nan     0.3896   -0.0960
##   2900       33.9943             nan     0.3896   -0.0507
##   2920       33.9042             nan     0.3896   -0.0412
##   2940       33.8041             nan     0.3896   -0.0979
##   2960       33.7884             nan     0.3896   -0.1181
##   2980       33.6237             nan     0.3896   -0.0506
##   3000       33.4821             nan     0.3896   -0.0732
##   3020       33.4736             nan     0.3896   -0.1440
##   3040       33.4127             nan     0.3896   -0.0753
##   3060       33.3178             nan     0.3896   -0.0322
##   3080       33.2760             nan     0.3896   -0.2064
##   3100       33.1716             nan     0.3896   -0.0657
##   3120       33.1371             nan     0.3896   -0.1217
##   3140       32.9607             nan     0.3896   -0.0618
##   3160       32.8767             nan     0.3896   -0.2242
##   3180       32.8222             nan     0.3896   -0.1226
##   3200       32.8001             nan     0.3896   -0.1425
##   3220       32.6850             nan     0.3896   -0.1300
##   3240       32.6208             nan     0.3896   -0.0329
##   3260       32.5128             nan     0.3896   -0.0512
##   3280       32.4430             nan     0.3896   -0.1764
##   3300       32.3674             nan     0.3896   -0.1104
##   3320       32.3272             nan     0.3896   -0.0456
##   3340       32.2889             nan     0.3896   -0.0825
##   3360       32.1970             nan     0.3896   -0.0818
##   3380       32.1566             nan     0.3896   -0.0593
##   3400       32.1000             nan     0.3896   -0.0829
##   3420       32.0453             nan     0.3896   -0.1719
##   3440       31.9537             nan     0.3896   -0.0599
##   3460       31.9128             nan     0.3896   -0.2856
##   3480       31.8516             nan     0.3896   -0.0653
##   3500       31.7987             nan     0.3896   -0.1114
##   3520       31.6687             nan     0.3896   -0.0457
##   3540       31.6340             nan     0.3896   -0.0677
##   3560       31.5510             nan     0.3896   -0.0599
##   3580       31.4923             nan     0.3896   -0.0924
##   3600       31.4246             nan     0.3896   -0.0682
##   3620       31.3499             nan     0.3896   -0.1036
##   3640       31.2919             nan     0.3896   -0.1012
##   3660       31.2134             nan     0.3896   -0.0649
##   3680       31.1513             nan     0.3896   -0.1660
##   3700       31.1048             nan     0.3896   -0.2067
##   3720       31.0415             nan     0.3896   -0.1493
##   3740       31.0269             nan     0.3896   -0.2850
##   3760       30.9477             nan     0.3896   -0.0107
##   3780       30.8868             nan     0.3896   -0.0735
##   3800       30.7940             nan     0.3896   -0.0767
##   3820       30.7705             nan     0.3896   -0.0397
##   3840       30.7294             nan     0.3896   -0.0413
##   3860       30.6902             nan     0.3896   -0.0963
##   3880       30.6040             nan     0.3896   -0.0383
##   3900       30.5430             nan     0.3896   -0.0772
##   3920       30.5378             nan     0.3896   -0.1499
##   3940       30.4630             nan     0.3896   -0.0624
##   3960       30.4285             nan     0.3896   -0.0625
##   3980       30.4085             nan     0.3896   -0.1534
##   4000       30.3510             nan     0.3896   -0.0691
##   4020       30.2927             nan     0.3896   -0.0372
##   4040       30.2426             nan     0.3896   -0.0861
##   4060       30.1657             nan     0.3896   -0.0670
##   4080       30.1677             nan     0.3896   -0.0924
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      542.3070             nan     0.5470  644.4317
##      2      368.4464             nan     0.5470  169.7810
##      3      314.3265             nan     0.5470   54.6140
##      4      290.2978             nan     0.5470   22.4680
##      5      271.5018             nan     0.5470   17.9091
##      6      259.1657             nan     0.5470   10.8291
##      7      250.7262             nan     0.5470    8.0476
##      8      240.5897             nan     0.5470    8.5867
##      9      232.6700             nan     0.5470    7.2057
##     10      229.2670             nan     0.5470    2.9190
##     20      193.5131             nan     0.5470    2.1525
##     40      161.0752             nan     0.5470    0.5130
##     60      143.5336             nan     0.5470    0.1071
##     80      131.8329             nan     0.5470   -0.2857
##    100      123.0206             nan     0.5470   -0.0716
##    120      116.1552             nan     0.5470   -0.0362
##    140      109.9345             nan     0.5470    0.1926
##    160      104.8267             nan     0.5470    0.1431
##    180      100.5709             nan     0.5470   -0.0387
##    200       97.4286             nan     0.5470   -0.3271
##    220       94.5115             nan     0.5470   -0.5509
##    240       91.2204             nan     0.5470   -0.0902
##    260       88.3644             nan     0.5470   -0.1467
##    280       85.8495             nan     0.5470   -0.1958
##    300       83.5324             nan     0.5470   -0.1665
##    320       81.4644             nan     0.5470   -0.3960
##    340       79.7604             nan     0.5470   -0.0978
##    360       78.1559             nan     0.5470   -0.1536
##    380       76.3635             nan     0.5470   -0.3493
##    400       74.4501             nan     0.5470   -0.2796
##    420       72.8697             nan     0.5470   -0.3021
##    440       71.3206             nan     0.5470   -0.2629
##    460       70.0988             nan     0.5470   -0.1268
##    480       68.7720             nan     0.5470   -0.2221
##    500       67.5797             nan     0.5470   -0.0668
##    520       66.5229             nan     0.5470   -0.1994
##    540       65.4074             nan     0.5470   -0.1591
##    560       64.3825             nan     0.5470   -0.2525
##    580       63.1884             nan     0.5470   -0.1506
##    600       62.1842             nan     0.5470   -0.1038
##    620       61.2088             nan     0.5470    0.0185
##    640       60.4141             nan     0.5470   -0.0729
##    660       59.6112             nan     0.5470   -0.1775
##    680       58.8692             nan     0.5470   -0.3279
##    700       58.0852             nan     0.5470   -0.1672
##    720       57.3576             nan     0.5470   -0.1369
##    740       56.5296             nan     0.5470   -0.1059
##    760       55.9631             nan     0.5470   -0.1367
##    780       55.4404             nan     0.5470   -0.2815
##    800       54.8895             nan     0.5470   -0.1879
##    820       54.0018             nan     0.5470   -0.0501
##    840       53.6397             nan     0.5470   -0.1549
##    860       53.0866             nan     0.5470   -0.1204
##    880       52.6250             nan     0.5470   -0.3210
##    900       51.9648             nan     0.5470   -0.1724
##    920       51.4949             nan     0.5470   -0.1707
##    940       50.9150             nan     0.5470   -0.1991
##    960       50.6005             nan     0.5470   -0.1655
##    980       50.1371             nan     0.5470   -0.1115
##   1000       49.6094             nan     0.5470   -0.1691
##   1020       49.1445             nan     0.5470   -0.0529
##   1040       48.7974             nan     0.5470   -0.0589
##   1060       48.4254             nan     0.5470   -0.2869
##   1080       48.1189             nan     0.5470   -0.2441
##   1100       47.6865             nan     0.5470   -0.1308
##   1120       47.2913             nan     0.5470   -0.1349
##   1140       46.8013             nan     0.5470   -0.1766
##   1160       46.7548             nan     0.5470   -0.0820
##   1180       46.2962             nan     0.5470   -0.1355
##   1200       46.1501             nan     0.5470   -0.1416
##   1220       45.9506             nan     0.5470   -0.1189
##   1240       45.5476             nan     0.5470   -0.2545
##   1260       45.1888             nan     0.5470   -0.1986
##   1280       44.9738             nan     0.5470   -0.2659
##   1300       44.5453             nan     0.5470    0.0091
##   1320       44.1553             nan     0.5470   -0.0806
##   1340       43.8385             nan     0.5470   -0.0776
##   1360       43.5605             nan     0.5470   -0.0254
##   1380       43.3799             nan     0.5470   -0.4751
##   1400       43.0405             nan     0.5470   -0.0629
##   1420       42.7333             nan     0.5470   -0.1647
##   1440       42.3977             nan     0.5470   -0.0333
##   1460       42.2396             nan     0.5470   -0.1350
##   1480       41.9200             nan     0.5470   -0.3009
##   1500       41.6715             nan     0.5470   -0.0658
##   1520       41.5951             nan     0.5470   -0.2379
##   1540       41.4340             nan     0.5470   -0.1704
##   1560       41.2784             nan     0.5470   -0.1565
##   1580       41.0612             nan     0.5470   -0.2533
##   1600       40.8879             nan     0.5470   -0.1364
##   1620       40.6279             nan     0.5470   -0.2405
##   1640       40.4515             nan     0.5470   -0.0808
##   1660       40.1216             nan     0.5470   -0.0216
##   1680       39.9889             nan     0.5470   -0.1354
##   1700       39.9035             nan     0.5470   -0.1826
##   1720       39.5808             nan     0.5470   -0.0906
##   1740       39.5627             nan     0.5470   -0.1038
##   1760       39.4605             nan     0.5470   -0.0630
##   1780       39.3401             nan     0.5470   -0.3842
##   1800       39.2815             nan     0.5470   -0.2805
##   1820       38.9755             nan     0.5470   -0.1910
##   1840       38.8255             nan     0.5470   -0.1247
##   1860       38.6672             nan     0.5470   -0.0948
##   1880       38.3884             nan     0.5470   -0.0536
##   1900       38.2184             nan     0.5470   -0.1244
##   1920       38.1689             nan     0.5470   -0.2206
##   1940       38.0343             nan     0.5470   -0.0785
##   1960       37.9118             nan     0.5470   -0.1620
##   1980       37.6805             nan     0.5470   -0.1042
##   2000       37.5742             nan     0.5470   -0.1149
##   2020       37.3648             nan     0.5470   -0.0607
##   2040       37.1868             nan     0.5470   -0.1555
##   2060       36.9664             nan     0.5470   -0.0938
##   2080       36.7324             nan     0.5470   -0.0859
##   2100       36.6836             nan     0.5470   -0.1191
##   2120       36.5359             nan     0.5470   -0.1499
##   2140       36.3803             nan     0.5470   -0.0909
##   2160       36.1621             nan     0.5470   -0.0565
##   2180       36.0219             nan     0.5470   -0.1781
##   2200       35.9661             nan     0.5470   -0.5330
##   2220       35.7435             nan     0.5470   -0.2108
##   2240       35.6967             nan     0.5470   -0.1849
##   2260       35.6375             nan     0.5470   -0.1349
##   2280       35.5614             nan     0.5470   -0.0648
##   2300       35.3967             nan     0.5470   -0.1883
##   2320       35.2976             nan     0.5470   -0.2238
##   2340       35.2129             nan     0.5470   -0.0572
##   2360       35.0857             nan     0.5470   -0.0937
##   2380       34.9542             nan     0.5470   -0.1301
##   2400       34.8235             nan     0.5470   -0.1073
##   2420       34.7509             nan     0.5470   -0.0557
##   2440       34.6844             nan     0.5470   -0.0720
##   2460       34.6298             nan     0.5470   -0.2208
##   2480       34.5822             nan     0.5470   -0.2594
##   2500       34.4290             nan     0.5470   -0.1676
##   2520       34.3626             nan     0.5470   -0.0657
##   2540       34.2781             nan     0.5470   -0.1690
##   2560       34.2299             nan     0.5470   -0.0764
##   2580       34.1203             nan     0.5470   -0.0327
##   2600       33.9658             nan     0.5470   -0.0627
##   2620       33.8997             nan     0.5470   -0.0979
##   2640       33.8462             nan     0.5470   -0.0764
##   2660       33.7370             nan     0.5470   -0.0954
##   2680       33.5634             nan     0.5470   -0.1037
##   2700       33.4722             nan     0.5470   -0.0976
##   2720       33.4428             nan     0.5470   -0.0932
##   2740       33.2766             nan     0.5470   -0.0669
##   2760       33.2764             nan     0.5470   -0.1596
##   2780       33.1801             nan     0.5470   -0.1805
##   2800       33.1706             nan     0.5470   -0.1152
##   2820       33.0487             nan     0.5470   -0.1551
##   2840       32.9630             nan     0.5470   -0.1115
##   2860       32.8965             nan     0.5470   -0.1768
##   2880       32.7566             nan     0.5470   -0.0684
##   2900       32.8322             nan     0.5470   -0.1971
##   2920       32.7676             nan     0.5470   -0.1966
##   2940       32.7351             nan     0.5470   -0.0640
##   2960       32.6732             nan     0.5470   -0.1380
##   2980       32.5712             nan     0.5470   -0.1326
##   3000       32.4169             nan     0.5470   -0.0455
##   3020       32.3694             nan     0.5470   -0.1154
##   3040       32.4450             nan     0.5470   -0.2046
##   3060       32.3691             nan     0.5470   -0.0721
##   3080       32.2814             nan     0.5470   -0.0554
##   3100       32.2736             nan     0.5470   -0.1928
##   3120       32.1975             nan     0.5470   -0.2064
##   3140       32.1136             nan     0.5470   -0.1993
##   3160       32.0175             nan     0.5470   -0.1553
##   3180       31.9881             nan     0.5470   -0.1636
##   3200       31.8310             nan     0.5470   -0.1240
##   3220       31.8190             nan     0.5470   -0.0703
##   3240       31.7363             nan     0.5470   -0.0993
##   3260       31.7272             nan     0.5470   -0.0916
##   3280       31.8131             nan     0.5470   -0.2832
##   3300       31.6560             nan     0.5470   -0.1187
##   3320       31.6394             nan     0.5470   -0.1642
##   3340       31.6659             nan     0.5470   -0.0913
##   3360       31.6340             nan     0.5470   -0.2551
##   3380       31.4976             nan     0.5470   -0.0999
##   3400       31.3762             nan     0.5470   -0.0962
##   3420       31.3078             nan     0.5470   -0.1735
##   3440       31.1764             nan     0.5470   -0.0921
##   3460       31.1662             nan     0.5470   -0.2471
##   3480       31.1405             nan     0.5470   -0.0582
##   3489       31.0313             nan     0.5470   -0.0522
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      821.3276             nan     0.2306  359.3408
##      2      600.9738             nan     0.2306  219.8959
##      3      460.3253             nan     0.2306  138.7767
##      4      372.6743             nan     0.2306   86.5394
##      5      318.0866             nan     0.2306   53.6244
##      6      280.0616             nan     0.2306   36.9895
##      7      251.1584             nan     0.2306   27.7687
##      8      230.8238             nan     0.2306   18.7941
##      9      216.7866             nan     0.2306   13.1819
##     10      207.0602             nan     0.2306    8.3991
##     20      163.1324             nan     0.2306    2.1658
##     40      131.7700             nan     0.2306    0.5595
##     60      115.2830             nan     0.2306   -0.0900
##     80      102.6829             nan     0.2306    0.2112
##    100       94.5790             nan     0.2306    0.0239
##    120       87.5823             nan     0.2306   -0.0845
##    140       82.2477             nan     0.2306   -0.2152
##    160       77.8893             nan     0.2306   -0.0167
##    180       74.1453             nan     0.2306   -0.2456
##    200       70.5234             nan     0.2306   -0.0206
##    220       67.4654             nan     0.2306   -0.1024
##    240       64.6399             nan     0.2306   -0.1322
##    260       62.1822             nan     0.2306   -0.1345
##    280       60.0635             nan     0.2306   -0.2072
##    300       57.9217             nan     0.2306   -0.1804
##    320       56.0681             nan     0.2306   -0.1069
##    340       54.4497             nan     0.2306   -0.1241
##    360       52.8719             nan     0.2306   -0.1624
##    380       51.5280             nan     0.2306   -0.1925
##    400       50.2007             nan     0.2306   -0.1414
##    420       49.0710             nan     0.2306   -0.1109
##    440       47.8819             nan     0.2306   -0.1300
##    460       46.7860             nan     0.2306    0.0172
##    480       45.6933             nan     0.2306   -0.1042
##    500       44.7527             nan     0.2306   -0.0893
##    520       43.8969             nan     0.2306   -0.1233
##    540       43.0154             nan     0.2306   -0.0938
##    560       42.1757             nan     0.2306   -0.1225
##    580       41.5056             nan     0.2306   -0.1356
##    600       40.9089             nan     0.2306   -0.0900
##    620       40.0992             nan     0.2306   -0.0755
##    640       39.4106             nan     0.2306   -0.1481
##    660       38.7655             nan     0.2306   -0.0154
##    680       38.1465             nan     0.2306   -0.1514
##    700       37.6731             nan     0.2306   -0.2110
##    720       37.1437             nan     0.2306   -0.0823
##    740       36.6343             nan     0.2306   -0.0714
##    760       36.2013             nan     0.2306   -0.1316
##    780       35.8551             nan     0.2306   -0.2566
##    800       35.4992             nan     0.2306   -0.1493
##    820       35.1056             nan     0.2306   -0.1832
##    840       34.6721             nan     0.2306   -0.1240
##    860       34.3447             nan     0.2306   -0.0989
##    880       33.9247             nan     0.2306   -0.0895
##    900       33.5514             nan     0.2306   -0.1365
##    920       33.2885             nan     0.2306   -0.1165
##    940       33.1227             nan     0.2306   -0.1423
##    960       32.8403             nan     0.2306   -0.0933
##    980       32.5868             nan     0.2306   -0.0878
##   1000       32.3950             nan     0.2306   -0.1110
##   1020       32.1584             nan     0.2306   -0.0874
##   1040       31.8960             nan     0.2306   -0.0693
##   1060       31.6905             nan     0.2306   -0.0807
##   1080       31.4874             nan     0.2306   -0.0493
##   1100       31.2477             nan     0.2306   -0.1203
##   1120       31.0578             nan     0.2306   -0.1281
##   1140       30.9094             nan     0.2306   -0.1814
##   1160       30.6055             nan     0.2306   -0.0216
##   1180       30.4484             nan     0.2306   -0.1130
##   1200       30.3480             nan     0.2306   -0.1480
##   1220       30.0757             nan     0.2306   -0.1390
##   1240       29.9736             nan     0.2306   -0.0968
##   1260       29.8186             nan     0.2306   -0.1288
##   1280       29.6686             nan     0.2306   -0.1193
##   1300       29.5137             nan     0.2306   -0.1118
##   1320       29.4115             nan     0.2306   -0.1184
##   1340       29.2279             nan     0.2306   -0.1556
##   1360       29.0795             nan     0.2306   -0.1155
##   1380       28.9351             nan     0.2306   -0.0774
##   1400       28.7856             nan     0.2306   -0.1238
##   1420       28.6727             nan     0.2306   -0.1252
##   1440       28.5775             nan     0.2306   -0.1711
##   1460       28.4776             nan     0.2306   -0.1192
##   1480       28.4216             nan     0.2306   -0.2796
##   1500       28.3189             nan     0.2306   -0.1458
##   1520       28.2425             nan     0.2306   -0.3081
##   1540       28.1180             nan     0.2306   -0.3040
##   1560       27.9131             nan     0.2306   -0.2382
##   1580       27.8117             nan     0.2306   -0.0584
##   1600       27.7041             nan     0.2306   -0.0846
##   1620       27.5870             nan     0.2306   -0.1009
##   1640       27.4971             nan     0.2306   -0.1263
##   1660       27.3889             nan     0.2306   -0.0867
##   1680       27.2859             nan     0.2306   -0.1151
##   1700       27.1732             nan     0.2306   -0.0785
##   1720       27.1091             nan     0.2306   -0.1300
##   1740       27.0780             nan     0.2306   -0.0984
##   1760       26.9646             nan     0.2306   -0.0605
##   1780       26.8318             nan     0.2306   -0.1066
##   1800       26.7576             nan     0.2306   -0.1019
##   1820       26.6844             nan     0.2306   -0.1053
##   1840       26.6453             nan     0.2306   -0.1630
##   1860       26.5627             nan     0.2306   -0.0971
##   1880       26.4985             nan     0.2306   -0.2263
##   1900       26.4268             nan     0.2306   -0.0818
##   1920       26.3996             nan     0.2306   -0.1320
##   1940       26.3506             nan     0.2306   -0.0988
##   1960       26.3046             nan     0.2306   -0.0680
##   1980       26.1787             nan     0.2306   -0.1133
##   2000       26.0859             nan     0.2306   -0.1469
##   2020       26.1298             nan     0.2306   -0.1239
##   2040       26.0477             nan     0.2306   -0.1247
##   2060       25.9427             nan     0.2306   -0.0734
##   2080       25.9105             nan     0.2306   -0.1723
##   2100       25.8732             nan     0.2306   -0.1463
##   2120       25.8305             nan     0.2306   -0.2124
##   2140       25.7787             nan     0.2306   -0.1266
##   2160       25.6507             nan     0.2306   -0.1174
##   2180       25.6593             nan     0.2306   -0.2485
##   2200       25.5332             nan     0.2306   -0.1312
##   2220       25.4667             nan     0.2306   -0.0913
##   2240       25.4175             nan     0.2306   -0.1287
##   2260       25.3485             nan     0.2306   -0.1188
##   2280       25.2882             nan     0.2306   -0.1097
##   2300       25.2328             nan     0.2306   -0.1159
##   2320       25.1848             nan     0.2306   -0.1273
##   2340       25.1040             nan     0.2306   -0.1248
##   2360       25.0788             nan     0.2306   -0.0993
##   2380       24.9987             nan     0.2306   -0.0883
##   2400       24.9645             nan     0.2306   -0.1105
##   2420       24.9213             nan     0.2306   -0.1001
##   2440       24.8627             nan     0.2306   -0.1023
##   2460       24.8351             nan     0.2306   -0.1412
##   2480       24.8188             nan     0.2306   -0.1868
##   2500       24.7693             nan     0.2306   -0.1497
##   2520       24.7641             nan     0.2306   -0.1357
##   2540       24.6546             nan     0.2306   -0.0896
##   2560       24.6410             nan     0.2306   -0.0583
##   2580       24.6273             nan     0.2306   -0.1648
##   2600       24.5793             nan     0.2306   -0.2057
##   2620       24.5329             nan     0.2306   -0.1209
##   2640       24.4869             nan     0.2306   -0.1038
##   2660       24.4552             nan     0.2306   -0.1387
##   2680       24.4255             nan     0.2306   -0.1795
##   2700       24.3308             nan     0.2306   -0.0570
##   2720       24.2750             nan     0.2306   -0.1012
##   2740       24.2412             nan     0.2306   -0.0967
##   2760       24.1883             nan     0.2306   -0.0974
##   2780       24.1515             nan     0.2306   -0.1412
##   2800       24.0884             nan     0.2306   -0.0207
##   2820       24.0719             nan     0.2306   -0.1365
##   2840       24.0523             nan     0.2306   -0.0712
##   2860       24.0244             nan     0.2306   -0.1552
##   2880       23.9892             nan     0.2306   -0.1515
##   2900       23.9778             nan     0.2306   -0.2321
##   2920       23.9419             nan     0.2306   -0.1755
##   2940       23.8970             nan     0.2306   -0.0948
##   2960       23.8516             nan     0.2306   -0.1394
##   2980       23.8075             nan     0.2306   -0.1650
##   3000       23.7923             nan     0.2306   -0.1693
##   3020       23.7382             nan     0.2306   -0.1186
##   3040       23.7196             nan     0.2306   -0.0865
##   3060       23.6355             nan     0.2306   -0.0840
##   3080       23.6224             nan     0.2306   -0.1374
##   3100       23.5921             nan     0.2306   -0.1078
##   3120       23.5343             nan     0.2306   -0.0601
##   3140       23.5309             nan     0.2306   -0.1842
##   3160       23.5316             nan     0.2306   -0.1278
##   3180       23.4955             nan     0.2306   -0.1127
##   3200       23.4582             nan     0.2306   -0.2115
##   3220       23.4536             nan     0.2306   -0.0731
##   3240       23.4456             nan     0.2306   -0.1514
##   3260       23.4634             nan     0.2306   -0.1580
##   3280       23.4093             nan     0.2306   -0.1692
##   3300       23.4322             nan     0.2306   -0.1611
##   3320       23.3961             nan     0.2306   -0.1979
##   3340       23.3659             nan     0.2306   -0.1000
##   3360       23.2976             nan     0.2306   -0.0943
##   3380       23.2648             nan     0.2306   -0.1334
##   3400       23.3039             nan     0.2306   -0.0970
##   3420       23.1956             nan     0.2306   -0.1461
##   3440       23.1778             nan     0.2306   -0.1615
##   3460       23.1878             nan     0.2306   -0.1024
##   3480       23.1336             nan     0.2306   -0.0868
##   3500       23.1256             nan     0.2306   -0.1345
##   3520       23.1319             nan     0.2306   -0.1503
##   3540       23.1050             nan     0.2306   -0.2077
##   3560       23.0862             nan     0.2306   -0.1774
##   3580       23.0154             nan     0.2306   -0.0965
##   3600       23.0157             nan     0.2306   -0.1314
##   3620       23.0188             nan     0.2306   -0.1554
##   3640       22.9422             nan     0.2306   -0.1125
##   3660       22.9363             nan     0.2306   -0.1066
##   3680       22.9579             nan     0.2306   -0.1537
##   3700       22.9607             nan     0.2306   -0.1180
##   3720       22.9389             nan     0.2306   -0.1488
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      676.8287             nan     0.3896  505.9714
##      2      467.9709             nan     0.3896  200.9572
##      3      373.1212             nan     0.3896   94.0872
##      4      322.0677             nan     0.3896   50.2540
##      5      298.3192             nan     0.3896   22.7675
##      6      278.9564             nan     0.3896   18.5713
##      7      267.4553             nan     0.3896   11.7286
##      8      258.4293             nan     0.3896    7.8794
##      9      249.7020             nan     0.3896    8.7866
##     10      243.4039             nan     0.3896    5.8770
##     20      204.9556             nan     0.3896    1.6074
##     40      171.7300             nan     0.3896    0.3857
##     60      154.7729             nan     0.3896    0.0859
##     80      141.1832             nan     0.3896    0.2308
##    100      131.9692             nan     0.3896    0.1222
##    120      124.8771             nan     0.3896    0.3768
##    140      119.6264             nan     0.3896    0.2125
##    160      114.5102             nan     0.3896    0.0814
##    180      110.3484             nan     0.3896   -0.1261
##    200      106.4667             nan     0.3896    0.0865
##    220      102.8844             nan     0.3896   -0.2331
##    240       99.7773             nan     0.3896   -0.1499
##    260       96.9362             nan     0.3896    0.0191
##    280       94.4762             nan     0.3896   -0.1258
##    300       92.6440             nan     0.3896   -0.2624
##    320       90.6197             nan     0.3896   -0.1723
##    340       88.8290             nan     0.3896   -0.1798
##    360       87.3736             nan     0.3896   -0.1580
##    380       85.4933             nan     0.3896   -0.0865
##    400       83.8877             nan     0.3896   -0.0746
##    420       82.1145             nan     0.3896   -0.0495
##    440       80.7048             nan     0.3896    0.0332
##    460       79.4790             nan     0.3896   -0.1341
##    480       78.5778             nan     0.3896   -0.1757
##    500       77.1777             nan     0.3896   -0.0136
##    520       76.0413             nan     0.3896   -0.0630
##    540       75.0136             nan     0.3896   -0.2771
##    560       73.9220             nan     0.3896   -0.1799
##    580       72.7934             nan     0.3896   -0.1094
##    600       71.9176             nan     0.3896   -0.0624
##    620       70.5897             nan     0.3896   -0.0914
##    640       69.6187             nan     0.3896   -0.1436
##    660       68.8592             nan     0.3896   -0.0062
##    680       68.0330             nan     0.3896   -0.0406
##    700       67.0629             nan     0.3896   -0.2016
##    720       66.2304             nan     0.3896   -0.1173
##    740       65.5563             nan     0.3896   -0.0605
##    760       64.5928             nan     0.3896   -0.0854
##    780       64.0018             nan     0.3896   -0.1374
##    800       63.3399             nan     0.3896   -0.1406
##    820       62.4860             nan     0.3896   -0.0692
##    840       61.9891             nan     0.3896   -0.0629
##    860       61.1159             nan     0.3896   -0.0343
##    880       60.5646             nan     0.3896   -0.0758
##    900       60.0431             nan     0.3896   -0.1246
##    920       59.3240             nan     0.3896   -0.1377
##    940       58.8028             nan     0.3896   -0.1204
##    960       58.1571             nan     0.3896   -0.0764
##    980       57.6320             nan     0.3896   -0.1426
##   1000       57.2725             nan     0.3896   -0.1796
##   1020       56.7767             nan     0.3896   -0.2723
##   1040       56.3310             nan     0.3896   -0.1124
##   1060       55.9910             nan     0.3896   -0.2214
##   1080       55.4359             nan     0.3896   -0.2438
##   1100       54.9728             nan     0.3896   -0.1382
##   1120       54.5233             nan     0.3896   -0.1839
##   1140       54.0115             nan     0.3896   -0.1016
##   1160       53.6043             nan     0.3896   -0.0981
##   1180       53.2291             nan     0.3896   -0.1246
##   1200       52.9151             nan     0.3896   -0.0252
##   1220       52.5999             nan     0.3896   -0.0890
##   1240       52.1093             nan     0.3896   -0.1119
##   1260       51.8856             nan     0.3896   -0.2350
##   1280       51.5313             nan     0.3896   -0.1463
##   1300       51.2772             nan     0.3896   -0.1585
##   1320       50.9728             nan     0.3896   -0.1015
##   1340       50.7128             nan     0.3896   -0.1269
##   1360       50.4766             nan     0.3896   -0.0812
##   1380       50.2285             nan     0.3896   -0.2744
##   1400       49.9840             nan     0.3896   -0.1105
##   1420       49.7098             nan     0.3896   -0.1211
##   1440       49.3795             nan     0.3896   -0.0909
##   1460       49.1345             nan     0.3896   -0.1356
##   1480       48.9698             nan     0.3896   -0.1806
##   1500       48.6178             nan     0.3896   -0.1611
##   1520       48.3375             nan     0.3896   -0.1153
##   1540       48.0315             nan     0.3896   -0.0840
##   1560       47.7846             nan     0.3896   -0.0894
##   1580       47.5932             nan     0.3896   -0.1034
##   1600       47.3227             nan     0.3896   -0.0601
##   1620       47.1003             nan     0.3896   -0.1954
##   1640       46.8264             nan     0.3896   -0.1180
##   1660       46.6298             nan     0.3896   -0.0623
##   1680       46.3883             nan     0.3896   -0.1209
##   1700       46.2756             nan     0.3896   -0.0784
##   1720       46.0333             nan     0.3896   -0.1305
##   1740       45.9145             nan     0.3896   -0.1290
##   1760       45.7022             nan     0.3896   -0.0923
##   1780       45.4369             nan     0.3896   -0.0985
##   1800       45.2338             nan     0.3896   -0.0775
##   1820       44.9947             nan     0.3896   -0.0811
##   1840       44.8071             nan     0.3896   -0.0899
##   1860       44.5211             nan     0.3896   -0.1377
##   1880       44.3113             nan     0.3896   -0.0899
##   1900       44.1861             nan     0.3896   -0.1104
##   1920       43.9825             nan     0.3896   -0.1269
##   1940       43.7299             nan     0.3896   -0.0252
##   1960       43.5450             nan     0.3896   -0.1480
##   1980       43.2532             nan     0.3896   -0.1382
##   2000       43.1142             nan     0.3896   -0.1281
##   2020       42.8611             nan     0.3896   -0.0760
##   2040       42.7771             nan     0.3896   -0.2003
##   2060       42.6444             nan     0.3896   -0.0250
##   2080       42.5149             nan     0.3896   -0.1468
##   2100       42.3439             nan     0.3896   -0.0920
##   2120       42.1556             nan     0.3896   -0.2042
##   2140       42.0129             nan     0.3896   -0.0920
##   2160       41.8834             nan     0.3896   -0.0705
##   2180       41.7364             nan     0.3896   -0.1326
##   2200       41.5859             nan     0.3896   -0.0704
##   2220       41.4262             nan     0.3896   -0.0961
##   2240       41.3361             nan     0.3896   -0.1172
##   2260       41.2428             nan     0.3896   -0.1088
##   2280       41.3340             nan     0.3896   -0.3704
##   2300       41.0908             nan     0.3896   -0.0875
##   2320       40.9103             nan     0.3896   -0.0691
##   2340       40.7466             nan     0.3896   -0.1031
##   2360       40.6143             nan     0.3896   -0.0597
##   2380       40.4672             nan     0.3896   -0.1510
##   2400       40.3150             nan     0.3896   -0.0669
##   2420       40.1931             nan     0.3896   -0.1276
##   2440       40.1820             nan     0.3896   -0.1633
##   2460       40.0486             nan     0.3896   -0.2673
##   2480       39.9413             nan     0.3896   -0.1272
##   2500       39.8281             nan     0.3896   -0.1158
##   2520       39.6440             nan     0.3896   -0.1168
##   2540       39.5155             nan     0.3896   -0.0856
##   2560       39.4196             nan     0.3896   -0.0708
##   2580       39.3138             nan     0.3896   -0.0960
##   2600       39.1487             nan     0.3896   -0.0573
##   2620       39.0594             nan     0.3896   -0.0649
##   2640       38.9706             nan     0.3896   -0.3441
##   2660       38.7677             nan     0.3896   -0.0537
##   2680       38.7314             nan     0.3896   -0.0902
##   2700       38.5691             nan     0.3896   -0.0750
##   2720       38.4278             nan     0.3896   -0.0386
##   2740       38.3359             nan     0.3896   -0.1170
##   2760       38.2188             nan     0.3896   -0.1087
##   2780       38.1124             nan     0.3896   -0.0449
##   2800       37.9610             nan     0.3896   -0.0090
##   2820       37.8863             nan     0.3896   -0.0674
##   2840       37.7188             nan     0.3896   -0.0977
##   2860       37.6552             nan     0.3896   -0.1190
##   2880       37.5271             nan     0.3896   -0.1117
##   2900       37.4721             nan     0.3896   -0.1001
##   2920       37.3468             nan     0.3896   -0.0524
##   2940       37.2613             nan     0.3896   -0.0907
##   2960       37.1944             nan     0.3896   -0.1266
##   2980       37.0689             nan     0.3896   -0.1222
##   3000       37.0133             nan     0.3896   -0.0981
##   3020       36.9193             nan     0.3896   -0.1368
##   3040       36.8685             nan     0.3896   -0.1411
##   3060       36.7645             nan     0.3896   -0.0748
##   3080       36.7170             nan     0.3896   -0.0859
##   3100       36.7008             nan     0.3896   -0.0898
##   3120       36.5408             nan     0.3896   -0.1598
##   3140       36.4719             nan     0.3896   -0.0814
##   3160       36.3800             nan     0.3896   -0.0698
##   3180       36.2939             nan     0.3896   -0.0961
##   3200       36.2154             nan     0.3896   -0.1578
##   3220       36.1782             nan     0.3896   -0.0864
##   3240       36.1416             nan     0.3896   -0.1789
##   3260       36.0564             nan     0.3896   -0.0554
##   3280       36.0000             nan     0.3896   -0.1661
##   3300       35.8914             nan     0.3896   -0.0586
##   3320       35.7453             nan     0.3896   -0.1078
##   3340       35.6428             nan     0.3896   -0.0919
##   3360       35.4969             nan     0.3896   -0.0816
##   3380       35.4574             nan     0.3896   -0.0926
##   3400       35.3927             nan     0.3896   -0.0857
##   3420       35.3836             nan     0.3896   -0.0902
##   3440       35.3137             nan     0.3896   -0.0589
##   3460       35.2626             nan     0.3896   -0.0031
##   3480       35.2074             nan     0.3896   -0.1331
##   3500       35.1717             nan     0.3896   -0.1036
##   3520       35.1323             nan     0.3896   -0.1364
##   3540       35.0813             nan     0.3896   -0.0705
##   3560       34.9507             nan     0.3896   -0.0687
##   3580       34.9035             nan     0.3896   -0.0818
##   3600       34.9124             nan     0.3896   -0.2005
##   3620       34.8530             nan     0.3896   -0.0495
##   3640       34.7578             nan     0.3896   -0.1039
##   3660       34.7142             nan     0.3896   -0.0516
##   3680       34.6825             nan     0.3896   -0.1496
##   3700       34.5440             nan     0.3896   -0.0473
##   3720       34.5356             nan     0.3896   -0.0817
##   3740       34.4491             nan     0.3896   -0.1598
##   3760       34.4218             nan     0.3896   -0.0919
##   3780       34.3640             nan     0.3896   -0.0673
##   3800       34.2794             nan     0.3896   -0.0990
##   3820       34.2235             nan     0.3896   -0.1252
##   3840       34.1632             nan     0.3896   -0.1456
##   3860       34.0918             nan     0.3896   -0.1276
##   3880       34.0270             nan     0.3896   -0.0561
##   3900       33.9516             nan     0.3896   -0.0563
##   3920       33.8951             nan     0.3896   -0.0384
##   3940       33.8385             nan     0.3896   -0.1006
##   3960       33.7992             nan     0.3896   -0.0723
##   3980       33.6824             nan     0.3896   -0.0624
##   4000       33.5855             nan     0.3896   -0.0838
##   4020       33.5784             nan     0.3896   -0.1084
##   4040       33.5179             nan     0.3896   -0.0573
##   4060       33.4657             nan     0.3896   -0.0838
##   4080       33.3659             nan     0.3896   -0.1464
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      543.2902             nan     0.5470  642.9670
##      2      373.2822             nan     0.5470  161.9944
##      3      313.2520             nan     0.5470   60.3204
##      4      289.9231             nan     0.5470   21.5888
##      5      272.8668             nan     0.5470   14.6496
##      6      259.6409             nan     0.5470   12.8673
##      7      250.7581             nan     0.5470    8.3044
##      8      242.4308             nan     0.5470    6.5270
##      9      235.7331             nan     0.5470    5.5650
##     10      229.3432             nan     0.5470    4.3093
##     20      195.5961             nan     0.5470    1.3428
##     40      163.9721             nan     0.5470    0.3904
##     60      146.2753             nan     0.5470    0.1279
##     80      133.3696             nan     0.5470    0.2029
##    100      125.5669             nan     0.5470   -0.1192
##    120      118.6052             nan     0.5470   -0.2355
##    140      112.7494             nan     0.5470   -0.3300
##    160      107.8631             nan     0.5470   -0.2354
##    180      104.1517             nan     0.5470   -0.6006
##    200      100.8986             nan     0.5470   -0.3900
##    220       97.5433             nan     0.5470   -0.5273
##    240       94.5163             nan     0.5470   -0.1667
##    260       92.1952             nan     0.5470   -0.1039
##    280       89.7928             nan     0.5470   -0.1748
##    300       87.5074             nan     0.5470   -0.3275
##    320       85.9551             nan     0.5470   -0.0629
##    340       83.7395             nan     0.5470   -0.1689
##    360       82.3494             nan     0.5470   -0.3428
##    380       80.6743             nan     0.5470   -0.2775
##    400       78.8831             nan     0.5470   -0.1820
##    420       77.3032             nan     0.5470   -0.0858
##    440       75.8846             nan     0.5470   -0.4080
##    460       74.7209             nan     0.5470   -0.1814
##    480       73.4075             nan     0.5470   -0.2105
##    500       72.2259             nan     0.5470   -0.1348
##    520       71.1456             nan     0.5470   -0.1489
##    540       70.1142             nan     0.5470   -0.1355
##    560       69.3827             nan     0.5470   -0.2710
##    580       68.4950             nan     0.5470   -0.2422
##    600       67.4828             nan     0.5470   -0.4141
##    620       66.8674             nan     0.5470   -0.2572
##    640       66.1788             nan     0.5470   -0.2728
##    660       65.2337             nan     0.5470   -0.3546
##    680       64.4266             nan     0.5470   -0.2667
##    700       63.7410             nan     0.5470   -0.1275
##    720       63.2553             nan     0.5470   -0.1793
##    740       62.4586             nan     0.5470   -0.2372
##    760       61.7030             nan     0.5470   -0.2507
##    780       61.0215             nan     0.5470   -0.2345
##    800       60.2494             nan     0.5470   -0.1153
##    820       59.7484             nan     0.5470   -0.2199
##    840       58.9687             nan     0.5470   -0.2443
##    860       58.6195             nan     0.5470   -0.1420
##    880       58.1017             nan     0.5470   -0.1680
##    900       57.5742             nan     0.5470   -0.2305
##    920       57.1509             nan     0.5470   -0.1647
##    940       56.6896             nan     0.5470   -0.2911
##    960       56.2129             nan     0.5470   -0.0576
##    980       55.9502             nan     0.5470   -0.2753
##   1000       55.5421             nan     0.5470   -0.0968
##   1020       55.0909             nan     0.5470   -0.2751
##   1040       54.6635             nan     0.5470   -0.2257
##   1060       54.3639             nan     0.5470   -0.0943
##   1080       53.9924             nan     0.5470   -0.1386
##   1100       53.6179             nan     0.5470   -0.1257
##   1120       53.1353             nan     0.5470   -0.0993
##   1140       52.7589             nan     0.5470   -0.3723
##   1160       52.4177             nan     0.5470   -0.0920
##   1180       52.1075             nan     0.5470   -0.4182
##   1200       51.7874             nan     0.5470   -0.1833
##   1220       51.3800             nan     0.5470   -0.3128
##   1240       50.8858             nan     0.5470   -0.0867
##   1260       50.6231             nan     0.5470   -0.1651
##   1280       50.4502             nan     0.5470   -0.2353
##   1300       50.1398             nan     0.5470   -0.1726
##   1320       49.8098             nan     0.5470   -0.0456
##   1340       49.5186             nan     0.5470   -0.2296
##   1360       49.2603             nan     0.5470   -0.1637
##   1380       48.8592             nan     0.5470   -0.2014
##   1400       48.5384             nan     0.5470   -0.1874
##   1420       48.2950             nan     0.5470   -0.2182
##   1440       47.9710             nan     0.5470   -0.1755
##   1460       47.8083             nan     0.5470   -0.2109
##   1480       47.6721             nan     0.5470   -0.1533
##   1500       47.2083             nan     0.5470   -0.0607
##   1520       47.0553             nan     0.5470   -0.1543
##   1540       46.8502             nan     0.5470   -0.1077
##   1560       46.6412             nan     0.5470   -0.0965
##   1580       46.5228             nan     0.5470   -0.1653
##   1600       46.2853             nan     0.5470   -0.1785
##   1620       46.0030             nan     0.5470   -0.0922
##   1640       45.7408             nan     0.5470   -0.0234
##   1660       45.4373             nan     0.5470   -0.1322
##   1680       45.3572             nan     0.5470   -0.3595
##   1700       45.0745             nan     0.5470   -0.0944
##   1720       44.8740             nan     0.5470   -0.0694
##   1740       44.8181             nan     0.5470   -0.1782
##   1760       44.6851             nan     0.5470   -0.1465
##   1780       44.4759             nan     0.5470   -0.1182
##   1800       44.3284             nan     0.5470   -0.1627
##   1820       44.2490             nan     0.5470   -0.1362
##   1840       44.0351             nan     0.5470   -0.1079
##   1860       43.8330             nan     0.5470   -0.1418
##   1880       43.5797             nan     0.5470   -0.1219
##   1900       43.4890             nan     0.5470   -0.1027
##   1920       43.3024             nan     0.5470   -0.1515
##   1940       43.1346             nan     0.5470   -0.1033
##   1960       43.0272             nan     0.5470   -0.1601
##   1980       42.8972             nan     0.5470   -0.0687
##   2000       42.7543             nan     0.5470   -0.0923
##   2020       42.5170             nan     0.5470   -0.0971
##   2040       42.3054             nan     0.5470   -0.0910
##   2060       42.0507             nan     0.5470   -0.1613
##   2080       41.9463             nan     0.5470   -0.1255
##   2100       41.8276             nan     0.5470   -0.2771
##   2120       41.6204             nan     0.5470   -0.1755
##   2140       41.5846             nan     0.5470   -0.2069
##   2160       41.3908             nan     0.5470   -0.2599
##   2180       41.1993             nan     0.5470   -0.1865
##   2200       40.8918             nan     0.5470   -0.0519
##   2220       40.8622             nan     0.5470   -0.1517
##   2240       40.6850             nan     0.5470   -0.0821
##   2260       40.5959             nan     0.5470   -0.3322
##   2280       40.2669             nan     0.5470   -0.0506
##   2300       40.2046             nan     0.5470   -0.1228
##   2320       40.0961             nan     0.5470   -0.2448
##   2340       40.0020             nan     0.5470   -0.2002
##   2360       40.0348             nan     0.5470   -0.1346
##   2380       40.0184             nan     0.5470   -0.2327
##   2400       39.9439             nan     0.5470   -0.1842
##   2420       39.7557             nan     0.5470   -0.1474
##   2440       39.5652             nan     0.5470   -0.1239
##   2460       39.4805             nan     0.5470   -0.2741
##   2480       39.5025             nan     0.5470   -0.2562
##   2500       39.3105             nan     0.5470   -0.1870
##   2520       39.1510             nan     0.5470   -0.1515
##   2540       39.0858             nan     0.5470   -0.1792
##   2560       38.9945             nan     0.5470   -0.2104
##   2580       38.6985             nan     0.5470   -0.1916
##   2600       38.6645             nan     0.5470   -0.2186
##   2620       38.4867             nan     0.5470   -0.0851
##   2640       38.3352             nan     0.5470   -0.1307
##   2660       38.2832             nan     0.5470   -0.0775
##   2680       38.1380             nan     0.5470   -0.2111
##   2700       38.0856             nan     0.5470   -0.1089
##   2720       38.0207             nan     0.5470   -0.3192
##   2740       37.9320             nan     0.5470   -0.0061
##   2760       37.8371             nan     0.5470   -0.1631
##   2780       37.7681             nan     0.5470   -0.1179
##   2800       37.7008             nan     0.5470   -0.1577
##   2820       37.6497             nan     0.5470   -0.2364
##   2840       37.5277             nan     0.5470   -0.1984
##   2860       37.4864             nan     0.5470   -0.2001
##   2880       37.3232             nan     0.5470   -0.1487
##   2900       37.1842             nan     0.5470   -0.0683
##   2920       37.1141             nan     0.5470   -0.1939
##   2940       37.0779             nan     0.5470   -0.1367
##   2960       37.1065             nan     0.5470   -0.3537
##   2980       37.0875             nan     0.5470   -0.2776
##   3000       36.9304             nan     0.5470   -0.0910
##   3020       36.8365             nan     0.5470   -0.1062
##   3040       36.6602             nan     0.5470   -0.1549
##   3060       36.6713             nan     0.5470   -0.2063
##   3080       36.5461             nan     0.5470   -0.1155
##   3100       36.5288             nan     0.5470   -0.1784
##   3120       36.4955             nan     0.5470   -0.1031
##   3140       36.3815             nan     0.5470   -0.1499
##   3160       36.3068             nan     0.5470   -0.1878
##   3180       36.4305             nan     0.5470   -0.1305
##   3200       36.2651             nan     0.5470   -0.1855
##   3220       36.2261             nan     0.5470   -0.2551
##   3240       36.2460             nan     0.5470   -0.2845
##   3260       36.1162             nan     0.5470   -0.1622
##   3280       36.0146             nan     0.5470   -0.1623
##   3300       35.9886             nan     0.5470   -0.1559
##   3320       35.9898             nan     0.5470   -0.2043
##   3340       35.8479             nan     0.5470   -0.1480
##   3360       35.8408             nan     0.5470   -0.0746
##   3380       35.6562             nan     0.5470   -0.0437
##   3400       35.5697             nan     0.5470   -0.1509
##   3420       35.5071             nan     0.5470   -0.1226
##   3440       35.4057             nan     0.5470   -0.2016
##   3460       35.3219             nan     0.5470   -0.1464
##   3480       35.2542             nan     0.5470   -0.1468
##   3489       35.2627             nan     0.5470   -0.2350
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      820.6194             nan     0.2306  353.8688
##      2      599.1583             nan     0.2306  217.2470
##      3      460.5915             nan     0.2306  138.5036
##      4      369.1621             nan     0.2306   88.4547
##      5      309.9012             nan     0.2306   56.9300
##      6      271.6850             nan     0.2306   37.6357
##      7      245.2320             nan     0.2306   25.4972
##      8      228.3411             nan     0.2306   15.3058
##      9      213.9316             nan     0.2306   13.3273
##     10      204.2950             nan     0.2306    8.5204
##     20      161.5396             nan     0.2306    1.7651
##     40      133.4623             nan     0.2306    0.7274
##     60      117.1551             nan     0.2306   -0.0256
##     80      105.1088             nan     0.2306   -0.1494
##    100       96.1503             nan     0.2306   -0.0269
##    120       89.3925             nan     0.2306   -0.0719
##    140       84.0294             nan     0.2306   -0.1394
##    160       79.7003             nan     0.2306   -0.0210
##    180       75.6240             nan     0.2306   -0.0973
##    200       71.5431             nan     0.2306   -0.0922
##    220       68.4637             nan     0.2306   -0.1434
##    240       65.3949             nan     0.2306   -0.1707
##    260       62.9546             nan     0.2306   -0.0553
##    280       60.6517             nan     0.2306   -0.0950
##    300       58.5007             nan     0.2306   -0.1367
##    320       56.6252             nan     0.2306   -0.1769
##    340       54.6718             nan     0.2306   -0.0940
##    360       52.9685             nan     0.2306   -0.0658
##    380       51.2805             nan     0.2306   -0.0203
##    400       50.1273             nan     0.2306   -0.0502
##    420       48.8870             nan     0.2306   -0.1049
##    440       47.7696             nan     0.2306   -0.1053
##    460       46.8493             nan     0.2306   -0.1287
##    480       45.7117             nan     0.2306   -0.1242
##    500       44.8263             nan     0.2306   -0.1519
##    520       43.8201             nan     0.2306   -0.1555
##    540       42.8759             nan     0.2306   -0.1274
##    560       42.1196             nan     0.2306   -0.1254
##    580       41.4654             nan     0.2306   -0.1650
##    600       40.8042             nan     0.2306   -0.2020
##    620       40.0755             nan     0.2306   -0.1853
##    640       39.4831             nan     0.2306   -0.1462
##    660       38.8501             nan     0.2306   -0.1288
##    680       38.3266             nan     0.2306   -0.1538
##    700       37.7959             nan     0.2306   -0.0576
##    720       37.2940             nan     0.2306   -0.1725
##    740       36.8893             nan     0.2306   -0.1297
##    760       36.3590             nan     0.2306   -0.1548
##    780       35.8777             nan     0.2306   -0.2197
##    800       35.5212             nan     0.2306   -0.1608
##    820       35.0515             nan     0.2306   -0.1663
##    840       34.6376             nan     0.2306   -0.1291
##    860       34.2711             nan     0.2306   -0.0653
##    880       33.9617             nan     0.2306   -0.0885
##    900       33.7386             nan     0.2306   -0.0681
##    920       33.3970             nan     0.2306   -0.1269
##    940       33.0884             nan     0.2306   -0.0888
##    960       32.8715             nan     0.2306   -0.0974
##    980       32.5560             nan     0.2306   -0.0833
##   1000       32.2511             nan     0.2306   -0.1256
##   1020       31.9856             nan     0.2306   -0.1033
##   1040       31.7123             nan     0.2306   -0.0611
##   1060       31.4901             nan     0.2306   -0.1139
##   1080       31.2426             nan     0.2306   -0.1382
##   1100       31.0234             nan     0.2306   -0.0596
##   1120       30.7606             nan     0.2306   -0.1187
##   1140       30.5245             nan     0.2306   -0.1529
##   1160       30.3948             nan     0.2306   -0.0725
##   1180       30.1445             nan     0.2306   -0.1676
##   1200       29.9077             nan     0.2306   -0.0736
##   1220       29.7746             nan     0.2306   -0.1501
##   1240       29.5787             nan     0.2306   -0.0625
##   1260       29.4524             nan     0.2306   -0.1050
##   1280       29.2022             nan     0.2306   -0.0740
##   1300       29.0184             nan     0.2306   -0.1378
##   1320       28.8193             nan     0.2306   -0.0367
##   1340       28.7192             nan     0.2306   -0.1775
##   1360       28.5234             nan     0.2306   -0.1312
##   1380       28.3592             nan     0.2306   -0.0806
##   1400       28.2234             nan     0.2306   -0.1832
##   1420       28.0449             nan     0.2306   -0.0678
##   1440       27.9516             nan     0.2306   -0.1192
##   1460       27.8408             nan     0.2306   -0.0580
##   1480       27.7067             nan     0.2306   -0.1908
##   1500       27.5488             nan     0.2306   -0.0931
##   1520       27.4258             nan     0.2306   -0.2191
##   1540       27.3084             nan     0.2306   -0.0845
##   1560       27.2389             nan     0.2306   -0.0935
##   1580       27.1365             nan     0.2306   -0.1586
##   1600       26.9924             nan     0.2306   -0.2679
##   1620       26.9321             nan     0.2306   -0.0944
##   1640       26.8169             nan     0.2306   -0.0948
##   1660       26.7395             nan     0.2306   -0.1229
##   1680       26.6600             nan     0.2306   -0.1464
##   1700       26.5451             nan     0.2306   -0.1251
##   1720       26.4734             nan     0.2306   -0.0913
##   1740       26.4018             nan     0.2306   -0.0413
##   1760       26.3240             nan     0.2306   -0.1701
##   1780       26.2329             nan     0.2306   -0.0909
##   1800       26.1518             nan     0.2306   -0.0868
##   1820       26.1005             nan     0.2306   -0.1193
##   1840       25.9644             nan     0.2306   -0.1858
##   1860       25.9329             nan     0.2306   -0.2324
##   1880       25.8601             nan     0.2306   -0.1522
##   1900       25.7788             nan     0.2306   -0.0746
##   1920       25.7110             nan     0.2306   -0.0837
##   1940       25.6263             nan     0.2306   -0.1669
##   1960       25.6022             nan     0.2306   -0.2031
##   1980       25.5498             nan     0.2306   -0.0747
##   2000       25.4881             nan     0.2306   -0.1205
##   2020       25.4198             nan     0.2306   -0.1501
##   2040       25.3321             nan     0.2306   -0.1091
##   2060       25.3483             nan     0.2306   -0.1233
##   2080       25.2751             nan     0.2306   -0.1638
##   2100       25.1988             nan     0.2306   -0.1019
##   2120       25.1271             nan     0.2306   -0.0669
##   2140       25.0705             nan     0.2306   -0.0881
##   2160       24.9968             nan     0.2306   -0.1178
##   2180       24.9626             nan     0.2306   -0.1475
##   2200       24.9035             nan     0.2306   -0.1245
##   2220       24.8270             nan     0.2306   -0.1577
##   2240       24.8116             nan     0.2306   -0.1813
##   2260       24.7574             nan     0.2306   -0.2410
##   2280       24.6536             nan     0.2306   -0.0455
##   2300       24.6072             nan     0.2306   -0.0927
##   2320       24.5196             nan     0.2306   -0.1021
##   2340       24.4471             nan     0.2306   -0.1662
##   2360       24.3710             nan     0.2306   -0.0817
##   2380       24.3157             nan     0.2306   -0.1340
##   2400       24.2798             nan     0.2306   -0.1065
##   2420       24.2078             nan     0.2306   -0.0971
##   2440       24.1877             nan     0.2306   -0.0973
##   2460       24.1814             nan     0.2306   -0.2632
##   2480       24.0947             nan     0.2306   -0.1100
##   2500       24.0599             nan     0.2306   -0.0619
##   2520       24.0449             nan     0.2306   -0.0418
##   2540       23.9912             nan     0.2306   -0.1012
##   2560       23.9605             nan     0.2306   -0.0675
##   2580       23.8798             nan     0.2306   -0.1198
##   2600       23.8217             nan     0.2306   -0.0890
##   2620       23.8019             nan     0.2306   -0.1085
##   2640       23.7905             nan     0.2306   -0.1211
##   2660       23.7463             nan     0.2306   -0.1934
##   2680       23.7228             nan     0.2306   -0.0635
##   2700       23.6900             nan     0.2306   -0.0781
##   2720       23.6591             nan     0.2306   -0.1247
##   2740       23.6356             nan     0.2306   -0.1673
##   2760       23.6248             nan     0.2306   -0.1023
##   2780       23.5742             nan     0.2306   -0.1595
##   2800       23.5516             nan     0.2306   -0.2022
##   2820       23.5146             nan     0.2306   -0.1024
##   2840       23.4404             nan     0.2306   -0.1018
##   2860       23.3906             nan     0.2306   -0.0990
##   2880       23.3689             nan     0.2306   -0.1056
##   2900       23.3536             nan     0.2306   -0.1049
##   2920       23.3225             nan     0.2306   -0.0685
##   2940       23.3121             nan     0.2306   -0.1098
##   2960       23.2444             nan     0.2306   -0.1059
##   2980       23.2215             nan     0.2306   -0.1258
##   3000       23.2125             nan     0.2306   -0.0950
##   3020       23.1764             nan     0.2306   -0.1731
##   3040       23.1885             nan     0.2306   -0.0953
##   3060       23.1645             nan     0.2306   -0.1524
##   3080       23.1262             nan     0.2306   -0.0670
##   3100       23.1659             nan     0.2306   -0.1602
##   3120       23.0794             nan     0.2306   -0.0874
##   3140       23.0536             nan     0.2306   -0.1239
##   3160       23.0068             nan     0.2306   -0.0622
##   3180       22.9841             nan     0.2306   -0.1110
##   3200       22.9964             nan     0.2306   -0.1033
##   3220       22.9546             nan     0.2306   -0.1292
##   3240       22.9308             nan     0.2306   -0.1357
##   3260       22.8933             nan     0.2306   -0.0912
##   3280       22.8573             nan     0.2306   -0.1191
##   3300       22.8943             nan     0.2306   -0.0955
##   3320       22.8265             nan     0.2306   -0.1671
##   3340       22.8042             nan     0.2306   -0.1781
##   3360       22.7992             nan     0.2306   -0.1349
##   3380       22.7449             nan     0.2306   -0.1123
##   3400       22.7664             nan     0.2306   -0.0718
##   3420       22.6948             nan     0.2306   -0.1102
##   3440       22.6788             nan     0.2306   -0.1006
##   3460       22.7039             nan     0.2306   -0.1320
##   3480       22.6540             nan     0.2306   -0.1541
##   3500       22.6096             nan     0.2306   -0.0761
##   3520       22.5975             nan     0.2306   -0.1482
##   3540       22.5583             nan     0.2306   -0.1008
##   3560       22.5509             nan     0.2306   -0.1259
##   3580       22.5035             nan     0.2306   -0.1317
##   3600       22.4361             nan     0.2306   -0.1085
##   3620       22.4121             nan     0.2306   -0.0846
##   3640       22.4368             nan     0.2306   -0.1233
##   3660       22.4090             nan     0.2306   -0.1152
##   3680       22.4103             nan     0.2306   -0.1800
##   3700       22.3624             nan     0.2306   -0.0964
##   3720       22.3273             nan     0.2306   -0.1325
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      677.9177             nan     0.3896  503.0452
##      2      464.4074             nan     0.3896  207.1565
##      3      367.1338             nan     0.3896   94.8912
##      4      316.2130             nan     0.3896   48.2976
##      5      289.9543             nan     0.3896   25.4996
##      6      272.4229             nan     0.3896   17.2176
##      7      261.1223             nan     0.3896   10.0322
##      8      249.5792             nan     0.3896   10.3391
##      9      242.6727             nan     0.3896    6.4213
##     10      237.2319             nan     0.3896    5.2651
##     20      203.0529             nan     0.3896    0.8312
##     40      167.4751             nan     0.3896    0.5045
##     60      151.7658             nan     0.3896    0.4061
##     80      142.7174             nan     0.3896   -0.0960
##    100      131.7149             nan     0.3896   -0.4479
##    120      124.9111             nan     0.3896   -0.1746
##    140      119.4816             nan     0.3896   -0.2282
##    160      113.9785             nan     0.3896    0.0066
##    180      108.4775             nan     0.3896   -0.0310
##    200      105.3022             nan     0.3896   -0.1651
##    220      101.8966             nan     0.3896    0.0017
##    240       98.6545             nan     0.3896   -0.0551
##    260       95.5319             nan     0.3896    0.1276
##    280       92.8357             nan     0.3896   -0.1542
##    300       90.5157             nan     0.3896    0.0413
##    320       88.6628             nan     0.3896   -0.1847
##    340       86.7293             nan     0.3896   -0.0754
##    360       84.5665             nan     0.3896   -0.1408
##    380       82.3831             nan     0.3896   -0.1413
##    400       80.8137             nan     0.3896   -0.1781
##    420       79.3110             nan     0.3896   -0.1592
##    440       77.7796             nan     0.3896   -0.1210
##    460       76.0648             nan     0.3896   -0.1209
##    480       74.8330             nan     0.3896   -0.3387
##    500       73.9394             nan     0.3896   -0.1504
##    520       72.9027             nan     0.3896   -0.0713
##    540       71.6474             nan     0.3896   -0.1716
##    560       70.7949             nan     0.3896   -0.2098
##    580       69.9202             nan     0.3896   -0.1876
##    600       68.6539             nan     0.3896   -0.1238
##    620       67.5875             nan     0.3896   -0.0925
##    640       66.7768             nan     0.3896   -0.1588
##    660       65.9534             nan     0.3896   -0.1482
##    680       65.4190             nan     0.3896   -0.1294
##    700       64.7703             nan     0.3896   -0.0468
##    720       63.8862             nan     0.3896   -0.1381
##    740       63.1045             nan     0.3896   -0.1360
##    760       62.1830             nan     0.3896   -0.1128
##    780       61.5404             nan     0.3896   -0.1327
##    800       60.8238             nan     0.3896   -0.1540
##    820       60.3089             nan     0.3896   -0.0917
##    840       59.7248             nan     0.3896   -0.1272
##    860       59.2553             nan     0.3896   -0.1269
##    880       58.6355             nan     0.3896   -0.1258
##    900       57.8617             nan     0.3896   -0.1275
##    920       57.4151             nan     0.3896   -0.1310
##    940       56.9016             nan     0.3896   -0.0943
##    960       56.2838             nan     0.3896   -0.0737
##    980       55.8246             nan     0.3896   -0.0323
##   1000       55.4061             nan     0.3896   -0.1532
##   1020       54.8312             nan     0.3896   -0.1318
##   1040       54.2491             nan     0.3896   -0.0682
##   1060       53.7525             nan     0.3896   -0.0769
##   1080       53.4079             nan     0.3896   -0.1129
##   1100       53.0612             nan     0.3896   -0.1347
##   1120       52.6730             nan     0.3896   -0.1217
##   1140       52.2650             nan     0.3896   -0.1253
##   1160       51.8112             nan     0.3896   -0.0734
##   1180       51.4796             nan     0.3896   -0.0611
##   1200       51.1854             nan     0.3896   -0.0803
##   1220       50.8764             nan     0.3896   -0.1109
##   1240       50.5035             nan     0.3896   -0.0951
##   1260       50.0948             nan     0.3896   -0.1251
##   1280       49.8616             nan     0.3896   -0.1204
##   1300       49.5378             nan     0.3896   -0.0765
##   1320       49.3096             nan     0.3896   -0.1237
##   1340       49.1052             nan     0.3896   -0.1456
##   1360       48.8655             nan     0.3896   -0.0673
##   1380       48.5866             nan     0.3896   -0.0879
##   1400       48.3641             nan     0.3896   -0.0813
##   1420       48.0880             nan     0.3896   -0.0542
##   1440       47.8252             nan     0.3896   -0.2488
##   1460       47.5270             nan     0.3896   -0.0655
##   1480       47.1983             nan     0.3896   -0.0625
##   1500       46.9476             nan     0.3896   -0.1421
##   1520       46.7118             nan     0.3896   -0.1146
##   1540       46.4980             nan     0.3896   -0.0695
##   1560       46.3585             nan     0.3896   -0.1489
##   1580       46.1005             nan     0.3896   -0.0866
##   1600       45.9274             nan     0.3896   -0.1146
##   1620       45.6893             nan     0.3896   -0.0693
##   1640       45.5171             nan     0.3896   -0.2473
##   1660       45.2703             nan     0.3896   -0.0935
##   1680       44.9753             nan     0.3896   -0.0486
##   1700       44.7373             nan     0.3896   -0.0949
##   1720       44.5174             nan     0.3896   -0.0627
##   1740       44.2596             nan     0.3896   -0.0338
##   1760       44.0570             nan     0.3896   -0.1186
##   1780       43.8737             nan     0.3896   -0.1193
##   1800       43.6546             nan     0.3896   -0.1226
##   1820       43.4366             nan     0.3896   -0.0900
##   1840       43.2918             nan     0.3896   -0.1056
##   1860       43.1475             nan     0.3896   -0.0921
##   1880       42.9396             nan     0.3896   -0.1200
##   1900       42.7686             nan     0.3896   -0.0842
##   1920       42.6692             nan     0.3896   -0.1257
##   1940       42.5785             nan     0.3896   -0.1466
##   1960       42.4136             nan     0.3896   -0.0571
##   1980       42.1970             nan     0.3896   -0.1208
##   2000       42.0065             nan     0.3896   -0.1136
##   2020       41.8058             nan     0.3896   -0.0495
##   2040       41.6862             nan     0.3896   -0.1610
##   2060       41.4721             nan     0.3896   -0.1532
##   2080       41.3383             nan     0.3896   -0.1159
##   2100       41.2395             nan     0.3896   -0.0968
##   2120       41.1132             nan     0.3896   -0.0456
##   2140       40.9744             nan     0.3896   -0.1031
##   2160       40.8291             nan     0.3896   -0.0861
##   2180       40.7293             nan     0.3896   -0.0371
##   2200       40.6236             nan     0.3896   -0.1067
##   2220       40.3859             nan     0.3896   -0.0989
##   2240       40.2353             nan     0.3896   -0.1386
##   2260       40.0893             nan     0.3896   -0.0829
##   2280       39.9074             nan     0.3896   -0.1228
##   2300       39.7626             nan     0.3896   -0.0912
##   2320       39.5553             nan     0.3896   -0.0859
##   2340       39.4032             nan     0.3896   -0.1333
##   2360       39.2632             nan     0.3896   -0.0876
##   2380       39.1619             nan     0.3896   -0.1540
##   2400       39.0062             nan     0.3896   -0.0680
##   2420       38.8738             nan     0.3896   -0.0326
##   2440       38.7165             nan     0.3896   -0.1705
##   2460       38.5941             nan     0.3896   -0.0764
##   2480       38.4791             nan     0.3896   -0.1146
##   2500       38.3235             nan     0.3896   -0.1291
##   2520       38.2607             nan     0.3896   -0.1004
##   2540       38.1519             nan     0.3896   -0.2056
##   2560       38.0500             nan     0.3896   -0.1051
##   2580       37.9584             nan     0.3896   -0.1498
##   2600       37.8551             nan     0.3896   -0.1124
##   2620       37.7084             nan     0.3896   -0.1415
##   2640       37.6301             nan     0.3896   -0.1817
##   2660       37.5307             nan     0.3896   -0.0623
##   2680       37.4721             nan     0.3896   -0.0416
##   2700       37.3431             nan     0.3896   -0.0776
##   2720       37.2182             nan     0.3896   -0.1071
##   2740       37.1750             nan     0.3896   -0.1129
##   2760       37.0388             nan     0.3896   -0.0593
##   2780       36.9265             nan     0.3896   -0.1085
##   2800       36.7457             nan     0.3896   -0.0617
##   2820       36.6568             nan     0.3896   -0.0830
##   2840       36.5762             nan     0.3896   -0.1197
##   2860       36.4910             nan     0.3896   -0.0622
##   2880       36.4077             nan     0.3896   -0.0708
##   2900       36.3286             nan     0.3896   -0.0702
##   2920       36.2666             nan     0.3896   -0.0767
##   2940       36.1699             nan     0.3896   -0.0991
##   2960       36.0668             nan     0.3896   -0.1080
##   2980       35.9766             nan     0.3896   -0.1736
##   3000       35.9440             nan     0.3896   -0.1042
##   3020       35.8180             nan     0.3896   -0.0545
##   3040       35.7321             nan     0.3896   -0.0259
##   3060       35.7266             nan     0.3896   -0.1374
##   3080       35.6597             nan     0.3896   -0.1294
##   3100       35.5163             nan     0.3896   -0.1079
##   3120       35.4309             nan     0.3896   -0.0681
##   3140       35.3129             nan     0.3896   -0.0977
##   3160       35.2468             nan     0.3896   -0.0670
##   3180       35.2879             nan     0.3896   -0.0850
##   3200       35.1955             nan     0.3896   -0.0702
##   3220       35.1500             nan     0.3896   -0.2726
##   3240       34.9397             nan     0.3896   -0.0804
##   3260       34.8954             nan     0.3896   -0.1351
##   3280       34.7431             nan     0.3896   -0.1590
##   3300       34.7291             nan     0.3896   -0.0467
##   3320       34.6046             nan     0.3896   -0.0695
##   3340       34.5626             nan     0.3896   -0.0980
##   3360       34.4333             nan     0.3896   -0.0471
##   3380       34.4596             nan     0.3896   -0.3348
##   3400       34.3071             nan     0.3896   -0.0517
##   3420       34.2082             nan     0.3896   -0.1041
##   3440       34.1743             nan     0.3896   -0.0279
##   3460       34.0891             nan     0.3896   -0.1009
##   3480       33.9961             nan     0.3896   -0.0494
##   3500       33.9499             nan     0.3896   -0.1065
##   3520       33.8591             nan     0.3896   -0.0638
##   3540       33.8026             nan     0.3896   -0.0811
##   3560       33.7864             nan     0.3896   -0.1999
##   3580       33.6565             nan     0.3896   -0.1060
##   3600       33.5864             nan     0.3896   -0.0740
##   3620       33.4546             nan     0.3896   -0.0873
##   3640       33.3822             nan     0.3896   -0.0860
##   3660       33.3406             nan     0.3896   -0.1075
##   3680       33.3233             nan     0.3896   -0.1265
##   3700       33.2399             nan     0.3896   -0.0770
##   3720       33.1669             nan     0.3896   -0.0627
##   3740       33.0661             nan     0.3896   -0.0745
##   3760       33.0417             nan     0.3896   -0.0460
##   3780       33.0633             nan     0.3896   -0.1175
##   3800       32.9575             nan     0.3896   -0.0781
##   3820       32.8900             nan     0.3896   -0.1539
##   3840       32.8047             nan     0.3896   -0.1122
##   3860       32.7430             nan     0.3896   -0.0562
##   3880       32.6737             nan     0.3896   -0.0858
##   3900       32.5778             nan     0.3896   -0.0104
##   3920       32.5213             nan     0.3896   -0.0919
##   3940       32.4966             nan     0.3896   -0.1097
##   3960       32.3621             nan     0.3896   -0.0910
##   3980       32.2821             nan     0.3896   -0.0674
##   4000       32.2862             nan     0.3896   -0.0861
##   4020       32.2191             nan     0.3896   -0.1623
##   4040       32.1737             nan     0.3896   -0.0448
##   4060       32.1100             nan     0.3896   -0.0370
##   4080       32.0268             nan     0.3896   -0.0451
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      545.4580             nan     0.5470  633.6213
##      2      377.1850             nan     0.5470  167.7014
##      3      323.5124             nan     0.5470   52.0036
##      4      292.3818             nan     0.5470   29.8596
##      5      277.8287             nan     0.5470   15.2389
##      6      265.7201             nan     0.5470   10.5742
##      7      258.1538             nan     0.5470    6.7848
##      8      250.2290             nan     0.5470    6.8402
##      9      244.8768             nan     0.5470    4.2257
##     10      237.5742             nan     0.5470    6.5635
##     20      198.3253             nan     0.5470    0.4298
##     40      164.5473             nan     0.5470    0.7642
##     60      148.2691             nan     0.5470   -0.1651
##     80      135.1759             nan     0.5470   -0.1611
##    100      127.8705             nan     0.5470   -0.2108
##    120      121.8714             nan     0.5470   -0.2966
##    140      116.0819             nan     0.5470   -0.2486
##    160      110.9791             nan     0.5470    0.1229
##    180      107.1375             nan     0.5470   -0.1261
##    200      103.5095             nan     0.5470   -0.2410
##    220       99.2128             nan     0.5470   -0.1087
##    240       96.3258             nan     0.5470    0.1416
##    260       93.7276             nan     0.5470   -0.3837
##    280       91.0612             nan     0.5470   -0.1789
##    300       88.5331             nan     0.5470    0.0414
##    320       86.4796             nan     0.5470   -0.2788
##    340       84.6957             nan     0.5470   -0.2004
##    360       83.0994             nan     0.5470   -0.3435
##    380       81.2186             nan     0.5470   -0.0541
##    400       79.3879             nan     0.5470   -0.0995
##    420       78.1239             nan     0.5470   -0.0574
##    440       77.0291             nan     0.5470   -0.1644
##    460       75.6788             nan     0.5470   -0.1747
##    480       74.4667             nan     0.5470   -0.3563
##    500       73.4982             nan     0.5470   -0.0722
##    520       72.4257             nan     0.5470   -0.1029
##    540       71.4823             nan     0.5470   -0.1524
##    560       70.4257             nan     0.5470   -0.3967
##    580       69.2170             nan     0.5470    0.0341
##    600       68.1516             nan     0.5470   -0.2551
##    620       67.1048             nan     0.5470   -0.1916
##    640       66.3775             nan     0.5470   -0.2300
##    660       65.4802             nan     0.5470   -0.0072
##    680       64.6658             nan     0.5470   -0.1738
##    700       63.9307             nan     0.5470   -0.1684
##    720       63.2211             nan     0.5470   -0.2351
##    740       62.5183             nan     0.5470   -0.0749
##    760       61.7917             nan     0.5470   -0.2708
##    780       61.0205             nan     0.5470   -0.3660
##    800       60.4357             nan     0.5470   -0.1108
##    820       59.7303             nan     0.5470   -0.2062
##    840       59.3639             nan     0.5470   -0.2726
##    860       58.8669             nan     0.5470   -0.3277
##    880       58.4731             nan     0.5470   -0.3554
##    900       57.8816             nan     0.5470   -0.0690
##    920       57.2542             nan     0.5470   -0.2311
##    940       56.6573             nan     0.5470   -0.4614
##    960       55.9429             nan     0.5470   -0.1096
##    980       55.5573             nan     0.5470   -0.1985
##   1000       55.1221             nan     0.5470   -0.2684
##   1020       54.5777             nan     0.5470   -0.1457
##   1040       54.1011             nan     0.5470   -0.2034
##   1060       53.6544             nan     0.5470   -0.0960
##   1080       53.3196             nan     0.5470   -0.1556
##   1100       52.9793             nan     0.5470   -0.1098
##   1120       52.6166             nan     0.5470   -0.1734
##   1140       52.0149             nan     0.5470   -0.1783
##   1160       51.8980             nan     0.5470   -0.2139
##   1180       51.2380             nan     0.5470   -0.2300
##   1200       50.6922             nan     0.5470   -0.0717
##   1220       50.3888             nan     0.5470   -0.2266
##   1240       49.8893             nan     0.5470   -0.0659
##   1260       49.6402             nan     0.5470   -0.1468
##   1280       49.3728             nan     0.5470   -0.2729
##   1300       49.1100             nan     0.5470   -0.1799
##   1320       48.6348             nan     0.5470   -0.0082
##   1340       48.2717             nan     0.5470   -0.1372
##   1360       48.1177             nan     0.5470   -0.2228
##   1380       47.8531             nan     0.5470   -0.1912
##   1400       47.6799             nan     0.5470   -0.1968
##   1420       47.4084             nan     0.5470   -0.1358
##   1440       47.0957             nan     0.5470   -0.0161
##   1460       46.9030             nan     0.5470   -0.2450
##   1480       46.6478             nan     0.5470   -0.2480
##   1500       46.2962             nan     0.5470   -0.1306
##   1520       46.0813             nan     0.5470   -0.0039
##   1540       45.9381             nan     0.5470   -0.1601
##   1560       45.7403             nan     0.5470   -0.1269
##   1580       45.7131             nan     0.5470   -0.5127
##   1600       45.4399             nan     0.5470   -0.2599
##   1620       45.3492             nan     0.5470   -0.3068
##   1640       45.0299             nan     0.5470   -0.1249
##   1660       44.7494             nan     0.5470   -0.1197
##   1680       44.5504             nan     0.5470   -0.1773
##   1700       44.4221             nan     0.5470   -0.1089
##   1720       44.2143             nan     0.5470   -0.2453
##   1740       44.0463             nan     0.5470   -0.1090
##   1760       43.9507             nan     0.5470   -0.1771
##   1780       43.7609             nan     0.5470   -0.1044
##   1800       43.5360             nan     0.5470   -0.1517
##   1820       43.3462             nan     0.5470   -0.2261
##   1840       43.1750             nan     0.5470   -0.2599
##   1860       43.1111             nan     0.5470   -0.2407
##   1880       42.8582             nan     0.5470   -0.0957
##   1900       42.7430             nan     0.5470   -0.2706
##   1920       42.6960             nan     0.5470   -0.1439
##   1940       42.5035             nan     0.5470   -0.2001
##   1960       42.2695             nan     0.5470   -0.2848
##   1980       41.9888             nan     0.5470   -0.1946
##   2000       41.8147             nan     0.5470   -0.1361
##   2020       41.7756             nan     0.5470   -0.3177
##   2040       41.6080             nan     0.5470   -0.1187
##   2060       41.5073             nan     0.5470   -0.0446
##   2080       41.3165             nan     0.5470   -0.0402
##   2100       41.2809             nan     0.5470   -0.1384
##   2120       41.1374             nan     0.5470   -0.0157
##   2140       40.8363             nan     0.5470    0.0062
##   2160       40.6662             nan     0.5470   -0.1969
##   2180       40.6572             nan     0.5470   -0.2372
##   2200       40.5961             nan     0.5470   -0.2361
##   2220       40.3117             nan     0.5470   -0.3391
##   2240       40.2932             nan     0.5470   -0.0424
##   2260       40.1921             nan     0.5470   -0.3075
##   2280       40.0839             nan     0.5470   -0.1916
##   2300       39.9071             nan     0.5470   -0.1374
##   2320       39.7727             nan     0.5470   -0.0791
##   2340       39.5200             nan     0.5470   -0.2150
##   2360       39.5038             nan     0.5470   -0.0720
##   2380       39.4039             nan     0.5470   -0.1439
##   2400       39.1900             nan     0.5470   -0.1554
##   2420       39.1232             nan     0.5470   -0.1278
##   2440       39.0439             nan     0.5470   -0.1986
##   2460       38.9547             nan     0.5470   -0.0981
##   2480       38.8996             nan     0.5470   -0.1513
##   2500       38.6764             nan     0.5470   -0.0622
##   2520       38.5672             nan     0.5470   -0.2582
##   2540       38.4975             nan     0.5470   -0.2054
##   2560       38.3661             nan     0.5470   -0.1409
##   2580       38.2186             nan     0.5470   -0.1450
##   2600       38.2406             nan     0.5470   -0.2135
##   2620       38.1130             nan     0.5470   -0.2169
##   2640       38.0224             nan     0.5470   -0.1717
##   2660       37.9413             nan     0.5470   -0.1680
##   2680       37.7852             nan     0.5470   -0.1288
##   2700       37.7605             nan     0.5470   -0.1656
##   2720       37.5747             nan     0.5470   -0.1121
##   2740       37.5366             nan     0.5470   -0.0994
##   2760       37.5062             nan     0.5470   -0.2266
##   2780       37.4467             nan     0.5470   -0.1813
##   2800       37.3650             nan     0.5470   -0.1997
##   2820       37.2200             nan     0.5470   -0.0712
##   2840       37.2164             nan     0.5470   -0.0906
##   2860       37.1314             nan     0.5470   -0.1702
##   2880       37.0179             nan     0.5470   -0.0824
##   2900       36.9730             nan     0.5470   -0.0944
##   2920       36.9626             nan     0.5470   -0.1693
##   2940       36.8526             nan     0.5470   -0.1317
##   2960       36.7494             nan     0.5470   -0.1241
##   2980       36.6386             nan     0.5470   -0.2117
##   3000       36.4818             nan     0.5470   -0.1311
##   3020       36.4702             nan     0.5470   -0.1226
##   3040       36.3716             nan     0.5470   -0.0688
##   3060       36.3829             nan     0.5470   -0.1010
##   3080       36.2960             nan     0.5470   -0.2946
##   3100       36.2825             nan     0.5470   -0.0936
##   3120       36.2139             nan     0.5470   -0.0881
##   3140       35.9991             nan     0.5470   -0.1436
##   3160       35.9321             nan     0.5470   -0.2617
##   3180       35.8465             nan     0.5470   -0.0206
##   3200       35.9516             nan     0.5470   -0.3832
##   3220       35.8693             nan     0.5470   -0.1734
##   3240       35.7836             nan     0.5470   -0.2704
##   3260       35.6468             nan     0.5470   -0.1090
##   3280       35.7024             nan     0.5470   -0.2618
##   3300       35.5157             nan     0.5470   -0.1845
##   3320       35.5796             nan     0.5470   -0.2319
##   3340       35.4250             nan     0.5470   -0.1013
##   3360       35.3250             nan     0.5470   -0.3907
##   3380       35.1500             nan     0.5470   -0.1760
##   3400       35.0896             nan     0.5470   -0.0542
##   3420       35.0693             nan     0.5470   -0.0700
##   3440       34.9707             nan     0.5470   -0.0592
##   3460       34.9264             nan     0.5470   -0.1814
##   3480       34.9231             nan     0.5470   -0.0963
##   3489       34.9143             nan     0.5470   -0.2436
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      817.9795             nan     0.2306  360.4836
##      2      598.4807             nan     0.2306  216.0674
##      3      461.0074             nan     0.2306  137.7353
##      4      374.8007             nan     0.2306   84.3834
##      5      317.1186             nan     0.2306   56.3556
##      6      277.3539             nan     0.2306   39.4376
##      7      247.3365             nan     0.2306   28.4586
##      8      228.7267             nan     0.2306   16.6954
##      9      216.3123             nan     0.2306   11.5026
##     10      206.0406             nan     0.2306    9.6836
##     20      166.0775             nan     0.2306    1.1928
##     40      134.6998             nan     0.2306    0.8234
##     60      118.6869             nan     0.2306    0.2444
##     80      107.4129             nan     0.2306    0.0591
##    100       98.0908             nan     0.2306    0.0024
##    120       91.4347             nan     0.2306   -0.0572
##    140       85.5194             nan     0.2306   -0.1464
##    160       80.8706             nan     0.2306   -0.0789
##    180       76.1076             nan     0.2306   -0.1694
##    200       72.6776             nan     0.2306   -0.1129
##    220       69.0522             nan     0.2306   -0.0469
##    240       66.4354             nan     0.2306   -0.1800
##    260       64.0556             nan     0.2306   -0.1068
##    280       61.4950             nan     0.2306   -0.0726
##    300       59.1289             nan     0.2306   -0.0969
##    320       56.9879             nan     0.2306   -0.2966
##    340       55.3250             nan     0.2306   -0.1203
##    360       53.6108             nan     0.2306   -0.1061
##    380       52.0894             nan     0.2306   -0.0931
##    400       50.8503             nan     0.2306   -0.1582
##    420       49.4024             nan     0.2306   -0.1012
##    440       48.3178             nan     0.2306   -0.2387
##    460       47.1026             nan     0.2306   -0.3775
##    480       46.1321             nan     0.2306   -0.2530
##    500       45.0794             nan     0.2306   -0.1855
##    520       44.1570             nan     0.2306   -0.0929
##    540       43.2673             nan     0.2306   -0.1313
##    560       42.4620             nan     0.2306   -0.1867
##    580       41.8434             nan     0.2306   -0.1059
##    600       41.2033             nan     0.2306   -0.0951
##    620       40.6021             nan     0.2306   -0.1737
##    640       39.9462             nan     0.2306   -0.1408
##    660       39.3580             nan     0.2306   -0.2850
##    680       38.8361             nan     0.2306   -0.0363
##    700       38.3530             nan     0.2306   -0.1149
##    720       37.9192             nan     0.2306   -0.0728
##    740       37.4982             nan     0.2306   -0.2226
##    760       37.0547             nan     0.2306   -0.1346
##    780       36.7159             nan     0.2306   -0.1924
##    800       36.2907             nan     0.2306   -0.1748
##    820       35.9679             nan     0.2306   -0.0885
##    840       35.6256             nan     0.2306   -0.0763
##    860       35.3323             nan     0.2306   -0.2201
##    880       34.9984             nan     0.2306   -0.2433
##    900       34.6927             nan     0.2306   -0.1291
##    920       34.3567             nan     0.2306   -0.0334
##    940       33.9907             nan     0.2306   -0.1187
##    960       33.7249             nan     0.2306   -0.1968
##    980       33.4330             nan     0.2306   -0.1714
##   1000       33.1904             nan     0.2306   -0.1833
##   1020       32.8853             nan     0.2306   -0.1060
##   1040       32.6740             nan     0.2306   -0.1740
##   1060       32.4728             nan     0.2306   -0.0738
##   1080       32.2581             nan     0.2306   -0.1734
##   1100       32.0569             nan     0.2306   -0.1095
##   1120       31.8486             nan     0.2306   -0.0841
##   1140       31.6796             nan     0.2306   -0.2219
##   1160       31.4127             nan     0.2306   -0.1248
##   1180       31.2262             nan     0.2306   -0.1120
##   1200       31.0011             nan     0.2306   -0.0515
##   1220       30.7591             nan     0.2306   -0.0954
##   1240       30.5947             nan     0.2306   -0.1652
##   1260       30.4199             nan     0.2306   -0.0787
##   1280       30.3480             nan     0.2306   -0.1435
##   1300       30.1607             nan     0.2306   -0.1145
##   1320       30.0036             nan     0.2306   -0.1797
##   1340       29.9179             nan     0.2306   -0.1277
##   1360       29.7753             nan     0.2306   -0.1056
##   1380       29.6477             nan     0.2306   -0.3715
##   1400       29.4705             nan     0.2306   -0.1181
##   1420       29.3166             nan     0.2306   -0.1385
##   1440       29.1990             nan     0.2306   -0.0919
##   1460       29.0493             nan     0.2306   -0.0725
##   1480       28.9642             nan     0.2306   -0.2413
##   1500       28.7937             nan     0.2306   -0.1754
##   1520       28.6657             nan     0.2306   -0.1000
##   1540       28.6247             nan     0.2306   -0.0675
##   1560       28.4601             nan     0.2306   -0.1913
##   1580       28.3916             nan     0.2306   -0.0345
##   1600       28.2613             nan     0.2306   -0.1151
##   1620       28.1973             nan     0.2306   -0.1869
##   1640       28.1115             nan     0.2306   -0.1793
##   1660       27.9913             nan     0.2306   -0.0616
##   1680       27.9009             nan     0.2306   -0.0896
##   1700       27.7669             nan     0.2306   -0.1711
##   1720       27.7085             nan     0.2306   -0.1217
##   1740       27.6306             nan     0.2306   -0.1984
##   1760       27.5403             nan     0.2306   -0.1664
##   1780       27.4948             nan     0.2306   -0.0983
##   1800       27.4132             nan     0.2306   -0.1426
##   1820       27.3641             nan     0.2306   -0.1237
##   1840       27.2890             nan     0.2306   -0.1091
##   1860       27.1503             nan     0.2306   -0.1135
##   1880       27.0451             nan     0.2306   -0.1433
##   1900       26.9583             nan     0.2306   -0.1342
##   1920       26.8906             nan     0.2306   -0.0999
##   1940       26.8026             nan     0.2306   -0.0798
##   1960       26.7496             nan     0.2306   -0.1644
##   1980       26.6784             nan     0.2306   -0.1353
##   2000       26.6389             nan     0.2306   -0.1009
##   2020       26.5524             nan     0.2306   -0.0735
##   2040       26.4746             nan     0.2306   -0.1330
##   2060       26.3416             nan     0.2306   -0.0355
##   2080       26.3365             nan     0.2306   -0.1825
##   2100       26.2533             nan     0.2306   -0.1313
##   2120       26.2615             nan     0.2306   -0.2340
##   2140       26.2220             nan     0.2306   -0.1737
##   2160       26.1153             nan     0.2306   -0.1919
##   2180       26.0397             nan     0.2306   -0.1721
##   2200       25.9755             nan     0.2306   -0.1409
##   2220       25.9346             nan     0.2306   -0.0954
##   2240       25.9222             nan     0.2306   -0.1267
##   2260       25.8269             nan     0.2306   -0.1524
##   2280       25.8248             nan     0.2306   -0.1873
##   2300       25.7486             nan     0.2306   -0.0751
##   2320       25.6771             nan     0.2306   -0.0946
##   2340       25.6173             nan     0.2306   -0.1312
##   2360       25.5190             nan     0.2306   -0.0862
##   2380       25.4710             nan     0.2306   -0.1422
##   2400       25.4149             nan     0.2306   -0.2781
##   2420       25.3470             nan     0.2306   -0.0874
##   2440       25.3217             nan     0.2306   -0.1568
##   2460       25.3369             nan     0.2306   -0.1782
##   2480       25.2815             nan     0.2306   -0.1958
##   2500       25.2057             nan     0.2306   -0.1389
##   2520       25.1317             nan     0.2306   -0.1332
##   2540       25.1060             nan     0.2306   -0.1213
##   2560       25.0587             nan     0.2306   -0.0810
##   2580       24.9746             nan     0.2306   -0.0494
##   2600       24.9371             nan     0.2306   -0.1664
##   2620       24.8844             nan     0.2306   -0.1293
##   2640       24.8802             nan     0.2306   -0.1153
##   2660       24.8383             nan     0.2306   -0.0795
##   2680       24.8175             nan     0.2306   -0.1591
##   2700       24.7614             nan     0.2306   -0.2255
##   2720       24.7698             nan     0.2306   -0.1399
##   2740       24.6850             nan     0.2306   -0.1258
##   2760       24.6717             nan     0.2306   -0.0685
##   2780       24.6360             nan     0.2306   -0.1243
##   2800       24.5752             nan     0.2306   -0.1358
##   2820       24.5738             nan     0.2306   -0.0812
##   2840       24.5310             nan     0.2306   -0.1090
##   2860       24.4829             nan     0.2306   -0.1870
##   2880       24.4522             nan     0.2306   -0.0816
##   2900       24.4477             nan     0.2306   -0.1055
##   2920       24.4027             nan     0.2306   -0.0524
##   2940       24.4237             nan     0.2306   -0.2255
##   2960       24.3622             nan     0.2306   -0.1191
##   2980       24.3827             nan     0.2306   -0.2022
##   3000       24.3112             nan     0.2306   -0.0965
##   3020       24.2653             nan     0.2306   -0.1439
##   3040       24.2794             nan     0.2306   -0.1860
##   3060       24.2194             nan     0.2306   -0.2111
##   3080       24.1806             nan     0.2306   -0.0871
##   3100       24.1661             nan     0.2306   -0.1039
##   3120       24.1212             nan     0.2306   -0.1003
##   3140       24.0893             nan     0.2306   -0.1213
##   3160       24.0823             nan     0.2306   -0.0947
##   3180       24.0364             nan     0.2306   -0.1161
##   3200       24.0435             nan     0.2306   -0.0843
##   3220       24.0398             nan     0.2306   -0.1800
##   3240       24.0091             nan     0.2306   -0.1425
##   3260       23.9417             nan     0.2306   -0.2095
##   3280       23.9026             nan     0.2306   -0.1323
##   3300       23.9140             nan     0.2306   -0.1541
##   3320       23.9296             nan     0.2306   -0.1896
##   3340       23.8719             nan     0.2306   -0.1582
##   3360       23.8423             nan     0.2306   -0.1819
##   3380       23.8938             nan     0.2306   -0.1150
##   3400       23.8465             nan     0.2306   -0.1298
##   3420       23.7941             nan     0.2306   -0.0677
##   3440       23.7690             nan     0.2306   -0.1067
##   3460       23.7706             nan     0.2306   -0.1912
##   3480       23.7022             nan     0.2306   -0.1122
##   3500       23.6862             nan     0.2306   -0.1415
##   3520       23.6781             nan     0.2306   -0.1864
##   3540       23.6432             nan     0.2306   -0.1977
##   3560       23.5986             nan     0.2306   -0.1682
##   3580       23.6527             nan     0.2306   -0.1548
##   3600       23.5695             nan     0.2306   -0.1224
##   3620       23.5275             nan     0.2306   -0.0994
##   3640       23.4952             nan     0.2306   -0.1615
##   3660       23.4934             nan     0.2306   -0.1831
##   3680       23.4672             nan     0.2306   -0.1061
##   3700       23.4580             nan     0.2306   -0.1143
##   3720       23.4243             nan     0.2306   -0.1221
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      676.0648             nan     0.3896  501.2356
##      2      468.6518             nan     0.3896  204.2582
##      3      369.5102             nan     0.3896   97.7376
##      4      320.0251             nan     0.3896   46.7955
##      5      293.5891             nan     0.3896   26.6887
##      6      278.3372             nan     0.3896   13.6571
##      7      266.9579             nan     0.3896   10.6189
##      8      256.4446             nan     0.3896    9.7409
##      9      247.6779             nan     0.3896    8.2827
##     10      242.7464             nan     0.3896    3.5754
##     20      205.1517             nan     0.3896    2.4785
##     40      172.9399             nan     0.3896    0.0750
##     60      155.8776             nan     0.3896    0.5588
##     80      144.2854             nan     0.3896    0.0818
##    100      134.4728             nan     0.3896    0.3944
##    120      128.0902             nan     0.3896   -0.0589
##    140      122.6520             nan     0.3896    0.2054
##    160      117.6196             nan     0.3896    0.0489
##    180      113.8429             nan     0.3896   -0.0327
##    200      109.9157             nan     0.3896   -0.1728
##    220      106.2275             nan     0.3896   -0.1117
##    240      103.3688             nan     0.3896   -0.2973
##    260      100.6896             nan     0.3896   -0.1908
##    280       97.8095             nan     0.3896   -0.0845
##    300       95.2219             nan     0.3896   -0.2201
##    320       92.8598             nan     0.3896   -0.3067
##    340       90.6615             nan     0.3896    0.0669
##    360       88.5973             nan     0.3896   -0.0997
##    380       85.9110             nan     0.3896   -0.1100
##    400       84.1636             nan     0.3896   -0.1650
##    420       82.7167             nan     0.3896   -0.1658
##    440       81.4345             nan     0.3896   -0.0900
##    460       79.7939             nan     0.3896   -0.0972
##    480       78.3835             nan     0.3896   -0.2399
##    500       77.0722             nan     0.3896   -0.1633
##    520       75.6129             nan     0.3896   -0.1140
##    540       74.6330             nan     0.3896   -0.0801
##    560       73.5004             nan     0.3896   -0.0417
##    580       72.4234             nan     0.3896   -0.2992
##    600       71.5520             nan     0.3896   -0.1255
##    620       70.7398             nan     0.3896   -0.1498
##    640       69.7438             nan     0.3896   -0.1730
##    660       68.8866             nan     0.3896   -0.0748
##    680       67.9188             nan     0.3896   -0.0662
##    700       66.9143             nan     0.3896   -0.3769
##    720       66.0884             nan     0.3896   -0.1911
##    740       65.3483             nan     0.3896   -0.0906
##    760       64.6753             nan     0.3896   -0.0700
##    780       64.0523             nan     0.3896   -0.1939
##    800       63.5281             nan     0.3896   -0.1176
##    820       62.8711             nan     0.3896   -0.1262
##    840       62.4436             nan     0.3896   -0.1009
##    860       61.8644             nan     0.3896   -0.0923
##    880       61.4104             nan     0.3896   -0.0945
##    900       60.7225             nan     0.3896   -0.1124
##    920       60.0694             nan     0.3896   -0.1315
##    940       59.5196             nan     0.3896   -0.1196
##    960       59.0759             nan     0.3896   -0.0833
##    980       58.5331             nan     0.3896   -0.0986
##   1000       58.1270             nan     0.3896   -0.1328
##   1020       57.6086             nan     0.3896   -0.0817
##   1040       57.1767             nan     0.3896   -0.0751
##   1060       56.6539             nan     0.3896   -0.1598
##   1080       56.3188             nan     0.3896   -0.1946
##   1100       55.6907             nan     0.3896   -0.1118
##   1120       55.3489             nan     0.3896   -0.0897
##   1140       54.8586             nan     0.3896   -0.2510
##   1160       54.5071             nan     0.3896   -0.1355
##   1180       54.2192             nan     0.3896   -0.0890
##   1200       53.8289             nan     0.3896   -0.1476
##   1220       53.4194             nan     0.3896   -0.1035
##   1240       52.8812             nan     0.3896   -0.1733
##   1260       52.3026             nan     0.3896   -0.0509
##   1280       51.8294             nan     0.3896   -0.1012
##   1300       51.4167             nan     0.3896   -0.1613
##   1320       50.9912             nan     0.3896   -0.1227
##   1340       50.5508             nan     0.3896   -0.1409
##   1360       50.2573             nan     0.3896   -0.1109
##   1380       49.9458             nan     0.3896   -0.1292
##   1400       49.6269             nan     0.3896   -0.0875
##   1420       49.4028             nan     0.3896   -0.2602
##   1440       48.9476             nan     0.3896   -0.0634
##   1460       48.7788             nan     0.3896   -0.1279
##   1480       48.5516             nan     0.3896   -0.0972
##   1500       48.1980             nan     0.3896   -0.1070
##   1520       47.9123             nan     0.3896   -0.1049
##   1540       47.5874             nan     0.3896   -0.1224
##   1560       47.3387             nan     0.3896   -0.1435
##   1580       47.1928             nan     0.3896   -0.0440
##   1600       46.8951             nan     0.3896   -0.0656
##   1620       46.7426             nan     0.3896   -0.2407
##   1640       46.4792             nan     0.3896   -0.1516
##   1660       46.1946             nan     0.3896   -0.2040
##   1680       46.0022             nan     0.3896   -0.1520
##   1700       45.7547             nan     0.3896   -0.0729
##   1720       45.4750             nan     0.3896   -0.1421
##   1740       45.2472             nan     0.3896   -0.0800
##   1760       44.9927             nan     0.3896   -0.0260
##   1780       44.8390             nan     0.3896   -0.1538
##   1800       44.6031             nan     0.3896   -0.0304
##   1820       44.3940             nan     0.3896   -0.0709
##   1840       44.2265             nan     0.3896   -0.0992
##   1860       44.0147             nan     0.3896   -0.0784
##   1880       43.8935             nan     0.3896   -0.1505
##   1900       43.6761             nan     0.3896   -0.0577
##   1920       43.5336             nan     0.3896   -0.1672
##   1940       43.3572             nan     0.3896   -0.0822
##   1960       43.0263             nan     0.3896   -0.0485
##   1980       42.8703             nan     0.3896   -0.1239
##   2000       42.7234             nan     0.3896   -0.0567
##   2020       42.6103             nan     0.3896   -0.1466
##   2040       42.5032             nan     0.3896   -0.1370
##   2060       42.3399             nan     0.3896   -0.1311
##   2080       42.1954             nan     0.3896   -0.2589
##   2100       42.0428             nan     0.3896   -0.1091
##   2120       41.8699             nan     0.3896   -0.1300
##   2140       41.7165             nan     0.3896   -0.0816
##   2160       41.6070             nan     0.3896   -0.0827
##   2180       41.4943             nan     0.3896   -0.0760
##   2200       41.3087             nan     0.3896   -0.0846
##   2220       41.0903             nan     0.3896   -0.0855
##   2240       41.0139             nan     0.3896   -0.0868
##   2260       40.8857             nan     0.3896   -0.1006
##   2280       40.7791             nan     0.3896   -0.0956
##   2300       40.6183             nan     0.3896   -0.1300
##   2320       40.4885             nan     0.3896   -0.1643
##   2340       40.4546             nan     0.3896   -0.1659
##   2360       40.2348             nan     0.3896   -0.1000
##   2380       40.0900             nan     0.3896   -0.1117
##   2400       39.8884             nan     0.3896   -0.1246
##   2420       39.8008             nan     0.3896   -0.1558
##   2440       39.6624             nan     0.3896   -0.1886
##   2460       39.6287             nan     0.3896   -0.0925
##   2480       39.4163             nan     0.3896   -0.0781
##   2500       39.4050             nan     0.3896   -0.1879
##   2520       39.3144             nan     0.3896   -0.0885
##   2540       39.1931             nan     0.3896   -0.1413
##   2560       39.0909             nan     0.3896   -0.1362
##   2580       38.9340             nan     0.3896   -0.0596
##   2600       38.8427             nan     0.3896   -0.1551
##   2620       38.7752             nan     0.3896   -0.1435
##   2640       38.5902             nan     0.3896   -0.0676
##   2660       38.4705             nan     0.3896   -0.0938
##   2680       38.3938             nan     0.3896   -0.1119
##   2700       38.3178             nan     0.3896   -0.0753
##   2720       38.2889             nan     0.3896   -0.0718
##   2740       38.1317             nan     0.3896   -0.0923
##   2760       38.0357             nan     0.3896   -0.0048
##   2780       37.9220             nan     0.3896   -0.0881
##   2800       37.8482             nan     0.3896   -0.0761
##   2820       37.7735             nan     0.3896   -0.0737
##   2840       37.7167             nan     0.3896   -0.1507
##   2860       37.6447             nan     0.3896   -0.0714
##   2880       37.5338             nan     0.3896   -0.0626
##   2900       37.4439             nan     0.3896   -0.1703
##   2920       37.3468             nan     0.3896   -0.0818
##   2940       37.1947             nan     0.3896   -0.0695
##   2960       37.1629             nan     0.3896   -0.1228
##   2980       37.0203             nan     0.3896   -0.0807
##   3000       36.9358             nan     0.3896   -0.1167
##   3020       36.8514             nan     0.3896   -0.0725
##   3040       36.7781             nan     0.3896   -0.1615
##   3060       36.7302             nan     0.3896   -0.0852
##   3080       36.6285             nan     0.3896   -0.0410
##   3100       36.4950             nan     0.3896   -0.0870
##   3120       36.3916             nan     0.3896   -0.0630
##   3140       36.3597             nan     0.3896   -0.1348
##   3160       36.2761             nan     0.3896   -0.0961
##   3180       36.1793             nan     0.3896   -0.0695
##   3200       36.0291             nan     0.3896   -0.0845
##   3220       35.9764             nan     0.3896   -0.1373
##   3240       35.8982             nan     0.3896   -0.0676
##   3260       35.7776             nan     0.3896   -0.0993
##   3280       35.7739             nan     0.3896   -0.1754
##   3300       35.7086             nan     0.3896   -0.0697
##   3320       35.6104             nan     0.3896    0.0074
##   3340       35.5593             nan     0.3896   -0.0887
##   3360       35.5020             nan     0.3896   -0.1140
##   3380       35.3907             nan     0.3896   -0.0652
##   3400       35.3092             nan     0.3896   -0.0825
##   3420       35.3100             nan     0.3896   -0.0978
##   3440       35.2294             nan     0.3896   -0.0198
##   3460       35.0980             nan     0.3896   -0.1147
##   3480       35.0338             nan     0.3896   -0.0846
##   3500       35.0240             nan     0.3896   -0.2173
##   3520       35.0554             nan     0.3896   -0.2055
##   3540       34.8980             nan     0.3896   -0.0909
##   3560       34.8176             nan     0.3896   -0.0812
##   3580       34.7459             nan     0.3896   -0.1058
##   3600       34.6967             nan     0.3896   -0.1857
##   3620       34.6003             nan     0.3896   -0.0169
##   3640       34.5725             nan     0.3896   -0.0997
##   3660       34.4810             nan     0.3896   -0.0399
##   3680       34.5286             nan     0.3896   -0.1458
##   3700       34.4055             nan     0.3896   -0.1054
##   3720       34.3393             nan     0.3896   -0.1668
##   3740       34.2928             nan     0.3896   -0.0690
##   3760       34.2423             nan     0.3896   -0.0924
##   3780       34.1979             nan     0.3896   -0.1044
##   3800       34.1073             nan     0.3896   -0.0942
##   3820       34.0990             nan     0.3896   -0.1416
##   3840       34.0286             nan     0.3896   -0.1006
##   3860       33.9801             nan     0.3896   -0.0731
##   3880       33.9531             nan     0.3896   -0.1806
##   3900       33.8676             nan     0.3896   -0.1078
##   3920       33.7365             nan     0.3896   -0.0843
##   3940       33.6904             nan     0.3896   -0.0980
##   3960       33.6399             nan     0.3896   -0.0983
##   3980       33.6576             nan     0.3896   -0.0506
##   4000       33.5392             nan     0.3896   -0.0368
##   4020       33.4918             nan     0.3896   -0.0692
##   4040       33.4557             nan     0.3896   -0.1096
##   4060       33.3201             nan     0.3896   -0.0838
##   4080       33.3455             nan     0.3896   -0.0729
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      541.6083             nan     0.5470  636.1436
##      2      370.0646             nan     0.5470  168.8557
##      3      313.7425             nan     0.5470   54.5681
##      4      285.8516             nan     0.5470   26.6381
##      5      270.1262             nan     0.5470   14.2004
##      6      256.9940             nan     0.5470   13.2382
##      7      247.4635             nan     0.5470    7.2631
##      8      240.0050             nan     0.5470    6.6923
##      9      233.7204             nan     0.5470    5.0025
##     10      225.9685             nan     0.5470    7.2794
##     20      193.2319             nan     0.5470    2.1870
##     40      163.2150             nan     0.5470    0.3537
##     60      146.1770             nan     0.5470   -0.7425
##     80      135.3933             nan     0.5470   -0.5240
##    100      124.2865             nan     0.5470   -0.2836
##    120      118.2953             nan     0.5470   -0.3208
##    140      112.4628             nan     0.5470    0.1064
##    160      107.7171             nan     0.5470   -0.2983
##    180      103.6233             nan     0.5470   -0.2360
##    200      100.1852             nan     0.5470   -0.2237
##    220       96.9017             nan     0.5470   -0.2497
##    240       93.8099             nan     0.5470   -0.1067
##    260       90.9776             nan     0.5470   -0.0615
##    280       88.2163             nan     0.5470   -0.2835
##    300       85.7009             nan     0.5470   -0.2298
##    320       83.9580             nan     0.5470   -0.1897
##    340       82.0331             nan     0.5470   -0.4068
##    360       80.2206             nan     0.5470   -0.2754
##    380       78.7930             nan     0.5470   -0.1792
##    400       77.7421             nan     0.5470   -0.3013
##    420       76.0578             nan     0.5470   -0.2647
##    440       74.8265             nan     0.5470   -0.3330
##    460       73.7321             nan     0.5470   -0.2232
##    480       72.6325             nan     0.5470   -0.1395
##    500       71.4685             nan     0.5470   -0.1924
##    520       70.1102             nan     0.5470   -0.0641
##    540       69.3563             nan     0.5470   -0.1142
##    560       68.2684             nan     0.5470   -0.2694
##    580       67.4660             nan     0.5470   -0.2834
##    600       66.7944             nan     0.5470   -0.3690
##    620       65.7721             nan     0.5470   -0.0915
##    640       64.8431             nan     0.5470   -0.2149
##    660       64.0885             nan     0.5470   -0.0629
##    680       63.2077             nan     0.5470   -0.0675
##    700       62.5557             nan     0.5470   -0.1196
##    720       61.9365             nan     0.5470   -0.0882
##    740       61.5223             nan     0.5470   -0.3251
##    760       60.7210             nan     0.5470   -0.2331
##    780       60.0930             nan     0.5470   -0.1335
##    800       59.5304             nan     0.5470   -0.1128
##    820       59.0534             nan     0.5470   -0.1825
##    840       58.7999             nan     0.5470   -0.1739
##    860       58.4868             nan     0.5470   -0.2244
##    880       57.9725             nan     0.5470   -0.2214
##    900       57.4016             nan     0.5470   -0.3069
##    920       56.7405             nan     0.5470   -0.1607
##    940       56.0748             nan     0.5470   -0.1881
##    960       55.6347             nan     0.5470   -0.2507
##    980       55.1061             nan     0.5470   -0.1446
##   1000       54.6796             nan     0.5470   -0.2802
##   1020       54.2648             nan     0.5470   -0.2905
##   1040       53.8657             nan     0.5470   -0.1711
##   1060       53.3997             nan     0.5470   -0.2236
##   1080       52.8751             nan     0.5470   -0.1636
##   1100       52.6996             nan     0.5470   -0.4565
##   1120       52.3388             nan     0.5470   -0.2636
##   1140       51.9756             nan     0.5470   -0.1102
##   1160       51.5908             nan     0.5470   -0.1930
##   1180       51.1381             nan     0.5470   -0.2381
##   1200       50.9780             nan     0.5470   -0.1290
##   1220       50.4341             nan     0.5470   -0.2706
##   1240       50.1026             nan     0.5470   -0.1748
##   1260       49.7954             nan     0.5470   -0.1334
##   1280       49.5121             nan     0.5470   -0.3638
##   1300       49.2868             nan     0.5470   -0.1547
##   1320       48.9974             nan     0.5470   -0.2079
##   1340       48.6562             nan     0.5470   -0.2062
##   1360       48.3412             nan     0.5470   -0.2585
##   1380       48.0411             nan     0.5470   -0.1828
##   1400       47.7449             nan     0.5470   -0.1628
##   1420       47.5158             nan     0.5470   -0.2113
##   1440       47.2465             nan     0.5470   -0.1194
##   1460       46.8845             nan     0.5470   -0.1525
##   1480       46.6141             nan     0.5470   -0.1405
##   1500       46.5144             nan     0.5470   -0.2423
##   1520       46.3309             nan     0.5470   -0.1459
##   1540       46.2831             nan     0.5470   -1.0062
##   1560       46.1378             nan     0.5470   -0.4530
##   1580       45.8304             nan     0.5470   -0.1792
##   1600       45.6366             nan     0.5470   -0.2370
##   1620       45.3521             nan     0.5470   -0.0818
##   1640       45.1041             nan     0.5470   -0.1231
##   1660       44.8855             nan     0.5470   -0.2310
##   1680       44.5277             nan     0.5470   -0.0878
##   1700       44.2718             nan     0.5470   -0.0579
##   1720       44.1424             nan     0.5470   -0.2465
##   1740       44.0150             nan     0.5470   -0.2961
##   1760       43.8585             nan     0.5470   -0.2561
##   1780       43.7131             nan     0.5470   -0.3731
##   1800       43.4632             nan     0.5470   -0.0801
##   1820       43.4333             nan     0.5470   -0.2216
##   1840       43.2563             nan     0.5470   -0.1434
##   1860       43.1481             nan     0.5470   -0.2650
##   1880       42.9628             nan     0.5470   -0.0593
##   1900       42.6539             nan     0.5470   -0.1161
##   1920       42.5111             nan     0.5470   -0.1866
##   1940       42.3627             nan     0.5470   -0.0603
##   1960       42.2901             nan     0.5470   -0.1565
##   1980       42.1250             nan     0.5470   -0.0666
##   2000       41.9995             nan     0.5470   -0.1283
##   2020       41.8516             nan     0.5470   -0.2224
##   2040       41.6227             nan     0.5470   -0.0353
##   2060       41.5567             nan     0.5470   -0.1596
##   2080       41.2611             nan     0.5470   -0.0057
##   2100       41.2371             nan     0.5470   -0.1911
##   2120       41.2002             nan     0.5470   -0.1603
##   2140       41.1717             nan     0.5470   -0.2339
##   2160       41.0124             nan     0.5470   -0.0785
##   2180       40.8616             nan     0.5470   -0.1504
##   2200       40.7149             nan     0.5470   -0.0667
##   2220       40.5354             nan     0.5470   -0.4046
##   2240       40.4184             nan     0.5470   -0.1981
##   2260       40.2478             nan     0.5470   -0.3158
##   2280       40.0977             nan     0.5470   -0.1932
##   2300       39.9658             nan     0.5470   -0.2602
##   2320       39.8949             nan     0.5470   -0.1310
##   2340       39.6920             nan     0.5470   -0.1954
##   2360       39.5991             nan     0.5470   -0.4759
##   2380       39.4466             nan     0.5470   -0.1083
##   2400       39.3770             nan     0.5470   -0.2319
##   2420       39.2103             nan     0.5470   -0.1624
##   2440       39.0828             nan     0.5470   -0.1379
##   2460       38.9707             nan     0.5470   -0.0916
##   2480       38.9498             nan     0.5470   -0.3318
##   2500       38.6598             nan     0.5470   -0.2023
##   2520       38.5696             nan     0.5470   -0.1426
##   2540       38.4622             nan     0.5470   -0.0932
##   2560       38.2634             nan     0.5470   -0.1747
##   2580       38.2333             nan     0.5470   -0.0531
##   2600       38.2137             nan     0.5470   -0.0999
##   2620       38.1811             nan     0.5470   -0.5628
##   2640       38.0447             nan     0.5470   -0.0684
##   2660       37.9239             nan     0.5470   -0.1771
##   2680       37.9042             nan     0.5470   -0.1053
##   2700       37.8045             nan     0.5470   -0.1121
##   2720       37.7525             nan     0.5470   -0.1981
##   2740       37.6252             nan     0.5470   -0.0256
##   2760       37.5571             nan     0.5470   -0.3739
##   2780       37.4810             nan     0.5470   -0.1618
##   2800       37.3772             nan     0.5470   -0.1383
##   2820       37.3748             nan     0.5470   -0.1305
##   2840       37.3636             nan     0.5470   -0.1884
##   2860       37.1688             nan     0.5470   -0.1112
##   2880       37.0890             nan     0.5470   -0.0930
##   2900       37.0410             nan     0.5470   -0.2062
##   2920       36.8339             nan     0.5470   -0.0909
##   2940       36.7895             nan     0.5470   -0.1916
##   2960       36.6864             nan     0.5470   -0.1507
##   2980       36.6446             nan     0.5470   -0.2444
##   3000       36.7085             nan     0.5470   -0.2183
##   3020       36.6667             nan     0.5470   -0.1746
##   3040       36.5575             nan     0.5470   -0.0687
##   3060       36.5145             nan     0.5470   -0.2578
##   3080       36.4743             nan     0.5470    0.0032
##   3100       36.3920             nan     0.5470   -0.1943
##   3120       36.2956             nan     0.5470   -0.0893
##   3140       36.2251             nan     0.5470   -0.0842
##   3160       36.2301             nan     0.5470   -0.1366
##   3180       36.0991             nan     0.5470   -0.1101
##   3200       35.9998             nan     0.5470   -0.3342
##   3220       35.8326             nan     0.5470   -0.1084
##   3240       35.8030             nan     0.5470   -0.2076
##   3260       35.8218             nan     0.5470   -0.1727
##   3280       35.7564             nan     0.5470   -0.1548
##   3300       35.6639             nan     0.5470   -0.0039
##   3320       35.6029             nan     0.5470   -0.1025
##   3340       35.5631             nan     0.5470   -0.0754
##   3360       35.4512             nan     0.5470   -0.2460
##   3380       35.3859             nan     0.5470   -0.1808
##   3400       35.3564             nan     0.5470   -0.3120
##   3420       35.4314             nan     0.5470   -0.2180
##   3440       35.2765             nan     0.5470   -0.1157
##   3460       35.1668             nan     0.5470   -0.1496
##   3480       35.1188             nan     0.5470   -0.1292
##   3489       35.1927             nan     0.5470   -0.1457
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      824.1791             nan     0.2306  358.2446
##      2      602.5942             nan     0.2306  223.1453
##      3      463.0799             nan     0.2306  135.0700
##      4      374.6483             nan     0.2306   87.0573
##      5      315.7440             nan     0.2306   57.9736
##      6      277.6372             nan     0.2306   37.3516
##      7      251.5559             nan     0.2306   25.0208
##      8      231.8101             nan     0.2306   19.4340
##      9      218.1287             nan     0.2306   12.3669
##     10      206.9774             nan     0.2306   10.6458
##     20      163.5534             nan     0.2306    1.4105
##     40      132.5967             nan     0.2306    0.3350
##     60      117.7354             nan     0.2306   -0.0359
##     80      106.8107             nan     0.2306   -0.1419
##    100       98.5681             nan     0.2306   -0.0179
##    120       90.9076             nan     0.2306   -0.0567
##    140       84.9125             nan     0.2306   -0.2862
##    160       80.1665             nan     0.2306   -0.2086
##    180       75.3929             nan     0.2306   -0.0920
##    200       71.2433             nan     0.2306    0.0180
##    220       67.9380             nan     0.2306   -0.1189
##    240       65.2977             nan     0.2306    0.0098
##    260       62.8858             nan     0.2306   -0.0623
##    280       60.4531             nan     0.2306   -0.0412
##    300       58.3719             nan     0.2306   -0.1843
##    320       56.5066             nan     0.2306   -0.1019
##    340       54.6092             nan     0.2306   -0.1770
##    360       52.9928             nan     0.2306   -0.1207
##    380       51.5212             nan     0.2306   -0.1014
##    400       49.9628             nan     0.2306   -0.1710
##    420       48.7485             nan     0.2306   -0.1628
##    440       47.4420             nan     0.2306   -0.1618
##    460       46.3054             nan     0.2306   -0.0631
##    480       45.3112             nan     0.2306   -0.0089
##    500       44.3890             nan     0.2306   -0.1116
##    520       43.4581             nan     0.2306   -0.2025
##    540       42.6000             nan     0.2306   -0.0255
##    560       41.8766             nan     0.2306   -0.1175
##    580       41.1541             nan     0.2306   -0.0486
##    600       40.4888             nan     0.2306   -0.1108
##    620       39.8739             nan     0.2306   -0.2493
##    640       39.2187             nan     0.2306   -0.0704
##    660       38.5169             nan     0.2306   -0.1071
##    680       37.8931             nan     0.2306   -0.0599
##    700       37.4024             nan     0.2306   -0.1801
##    720       36.8007             nan     0.2306   -0.0723
##    740       36.2326             nan     0.2306   -0.1152
##    760       35.8090             nan     0.2306   -0.2280
##    780       35.3867             nan     0.2306   -0.2246
##    800       34.9869             nan     0.2306   -0.1302
##    820       34.4502             nan     0.2306   -0.1088
##    840       34.1544             nan     0.2306   -0.0968
##    860       33.7443             nan     0.2306   -0.1576
##    880       33.3940             nan     0.2306   -0.1569
##    900       33.1384             nan     0.2306   -0.1269
##    920       32.7725             nan     0.2306   -0.2084
##    940       32.5182             nan     0.2306   -0.1493
##    960       32.2364             nan     0.2306   -0.1101
##    980       31.9601             nan     0.2306   -0.0781
##   1000       31.6693             nan     0.2306   -0.0604
##   1020       31.3130             nan     0.2306   -0.0403
##   1040       31.1118             nan     0.2306   -0.1073
##   1060       30.8807             nan     0.2306   -0.0722
##   1080       30.6953             nan     0.2306   -0.1296
##   1100       30.5683             nan     0.2306   -0.1063
##   1120       30.2876             nan     0.2306   -0.1496
##   1140       30.0851             nan     0.2306   -0.0926
##   1160       29.9268             nan     0.2306   -0.0386
##   1180       29.7517             nan     0.2306   -0.0970
##   1200       29.5675             nan     0.2306   -0.0740
##   1220       29.4328             nan     0.2306   -0.1279
##   1240       29.2892             nan     0.2306   -0.1031
##   1260       29.1074             nan     0.2306   -0.1611
##   1280       28.9510             nan     0.2306   -0.1357
##   1300       28.8129             nan     0.2306   -0.1868
##   1320       28.6876             nan     0.2306   -0.2699
##   1340       28.5318             nan     0.2306   -0.1583
##   1360       28.3877             nan     0.2306   -0.1315
##   1380       28.2421             nan     0.2306   -0.1018
##   1400       28.0664             nan     0.2306   -0.0826
##   1420       27.9392             nan     0.2306   -0.1684
##   1440       27.7951             nan     0.2306   -0.1274
##   1460       27.7004             nan     0.2306   -0.1123
##   1480       27.5709             nan     0.2306   -0.1310
##   1500       27.4284             nan     0.2306   -0.1343
##   1520       27.3364             nan     0.2306   -0.0552
##   1540       27.1638             nan     0.2306   -0.1070
##   1560       27.0503             nan     0.2306   -0.1075
##   1580       26.9914             nan     0.2306   -0.1209
##   1600       26.8289             nan     0.2306   -0.1151
##   1620       26.7256             nan     0.2306   -0.1090
##   1640       26.6368             nan     0.2306   -0.1753
##   1660       26.5661             nan     0.2306   -0.2066
##   1680       26.4189             nan     0.2306   -0.0974
##   1700       26.3229             nan     0.2306   -0.1110
##   1720       26.1898             nan     0.2306   -0.1314
##   1740       26.1169             nan     0.2306   -0.0546
##   1760       26.0244             nan     0.2306   -0.0942
##   1780       25.9758             nan     0.2306   -0.0654
##   1800       25.9276             nan     0.2306   -0.1488
##   1820       25.8199             nan     0.2306   -0.0876
##   1840       25.7263             nan     0.2306   -0.0909
##   1860       25.6422             nan     0.2306   -0.1357
##   1880       25.5902             nan     0.2306   -0.0815
##   1900       25.4785             nan     0.2306   -0.0981
##   1920       25.3365             nan     0.2306   -0.1041
##   1940       25.3208             nan     0.2306   -0.1441
##   1960       25.2230             nan     0.2306   -0.1627
##   1980       25.1191             nan     0.2306   -0.0809
##   2000       25.0568             nan     0.2306   -0.0538
##   2020       25.0206             nan     0.2306   -0.0752
##   2040       25.0037             nan     0.2306   -0.1084
##   2060       24.9322             nan     0.2306   -0.1599
##   2080       24.8867             nan     0.2306   -0.1490
##   2100       24.8245             nan     0.2306   -0.1460
##   2120       24.7587             nan     0.2306   -0.1150
##   2140       24.6998             nan     0.2306   -0.1222
##   2160       24.7084             nan     0.2306   -0.1039
##   2180       24.6559             nan     0.2306   -0.0815
##   2200       24.6070             nan     0.2306   -0.1216
##   2220       24.5250             nan     0.2306   -0.0850
##   2240       24.4679             nan     0.2306   -0.0874
##   2260       24.4272             nan     0.2306   -0.1140
##   2280       24.3430             nan     0.2306   -0.1383
##   2300       24.2886             nan     0.2306   -0.1095
##   2320       24.2214             nan     0.2306   -0.0867
##   2340       24.1815             nan     0.2306   -0.1020
##   2360       24.1820             nan     0.2306   -0.0683
##   2380       24.1362             nan     0.2306   -0.0607
##   2400       24.1275             nan     0.2306   -0.1380
##   2420       24.0435             nan     0.2306   -0.0954
##   2440       24.0182             nan     0.2306   -0.1505
##   2460       23.9880             nan     0.2306   -0.1150
##   2480       23.9417             nan     0.2306   -0.1621
##   2500       23.8582             nan     0.2306   -0.0792
##   2520       23.8636             nan     0.2306   -0.0661
##   2540       23.8234             nan     0.2306   -0.1018
##   2560       23.8074             nan     0.2306   -0.1665
##   2580       23.7704             nan     0.2306   -0.0781
##   2600       23.7107             nan     0.2306   -0.1177
##   2620       23.6841             nan     0.2306   -0.1349
##   2640       23.6538             nan     0.2306   -0.1020
##   2660       23.6234             nan     0.2306   -0.1328
##   2680       23.6007             nan     0.2306   -0.1080
##   2700       23.5655             nan     0.2306   -0.0790
##   2720       23.5933             nan     0.2306   -0.1217
##   2740       23.5512             nan     0.2306   -0.1216
##   2760       23.4836             nan     0.2306   -0.0760
##   2780       23.4662             nan     0.2306   -0.0789
##   2800       23.4250             nan     0.2306   -0.1772
##   2820       23.4035             nan     0.2306   -0.0981
##   2840       23.3607             nan     0.2306   -0.1186
##   2860       23.3284             nan     0.2306   -0.1818
##   2880       23.2648             nan     0.2306   -0.0726
##   2900       23.2493             nan     0.2306   -0.3566
##   2920       23.2352             nan     0.2306   -0.1454
##   2940       23.2144             nan     0.2306   -0.0986
##   2960       23.1937             nan     0.2306   -0.0847
##   2980       23.1873             nan     0.2306   -0.2321
##   3000       23.1565             nan     0.2306   -0.0832
##   3020       23.1238             nan     0.2306   -0.1313
##   3040       23.0693             nan     0.2306   -0.0959
##   3060       23.0845             nan     0.2306   -0.1525
##   3080       23.0624             nan     0.2306   -0.0978
##   3100       23.0172             nan     0.2306   -0.0985
##   3120       22.9917             nan     0.2306   -0.1493
##   3140       22.9472             nan     0.2306   -0.0827
##   3160       22.9172             nan     0.2306   -0.1525
##   3180       22.9283             nan     0.2306   -0.1023
##   3200       22.8934             nan     0.2306   -0.1494
##   3220       22.8558             nan     0.2306   -0.0613
##   3240       22.8180             nan     0.2306   -0.1042
##   3260       22.8153             nan     0.2306   -0.1170
##   3280       22.7372             nan     0.2306   -0.2528
##   3300       22.6994             nan     0.2306   -0.0853
##   3320       22.7109             nan     0.2306   -0.1543
##   3340       22.6615             nan     0.2306   -0.1434
##   3360       22.6107             nan     0.2306   -0.2297
##   3380       22.6079             nan     0.2306   -0.1744
##   3400       22.5634             nan     0.2306   -0.0462
##   3420       22.5848             nan     0.2306   -0.1517
##   3440       22.5851             nan     0.2306   -0.0564
##   3460       22.5221             nan     0.2306   -0.1324
##   3480       22.5139             nan     0.2306   -0.1425
##   3500       22.4823             nan     0.2306   -0.1276
##   3520       22.4176             nan     0.2306   -0.0875
##   3540       22.3976             nan     0.2306   -0.0774
##   3560       22.4229             nan     0.2306   -0.0791
##   3580       22.4124             nan     0.2306   -0.1216
##   3600       22.4153             nan     0.2306   -0.1153
##   3620       22.3811             nan     0.2306   -0.1968
##   3640       22.3235             nan     0.2306   -0.0789
##   3660       22.3481             nan     0.2306   -0.1513
##   3680       22.2957             nan     0.2306   -0.0698
##   3700       22.2775             nan     0.2306   -0.1331
##   3720       22.2341             nan     0.2306   -0.0996
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      671.3874             nan     0.3896  496.5516
##      2      465.5133             nan     0.3896  202.5717
##      3      372.1334             nan     0.3896   91.3579
##      4      321.7179             nan     0.3896   50.1235
##      5      295.4187             nan     0.3896   26.5645
##      6      278.8101             nan     0.3896   15.6311
##      7      266.4982             nan     0.3896   10.6099
##      8      259.1192             nan     0.3896    6.6450
##      9      251.3469             nan     0.3896    6.9884
##     10      245.6628             nan     0.3896    4.8410
##     20      203.1887             nan     0.3896    1.1346
##     40      169.2359             nan     0.3896    0.1007
##     60      152.8039             nan     0.3896    0.2980
##     80      143.0199             nan     0.3896   -0.2587
##    100      133.9578             nan     0.3896   -0.0136
##    120      127.3904             nan     0.3896   -0.3946
##    140      120.8181             nan     0.3896    0.1822
##    160      115.7616             nan     0.3896    0.0565
##    180      111.9011             nan     0.3896   -0.1597
##    200      108.2999             nan     0.3896   -0.0946
##    220      104.3543             nan     0.3896    0.0802
##    240      100.9133             nan     0.3896   -0.0854
##    260       98.1068             nan     0.3896   -0.2004
##    280       95.3875             nan     0.3896    0.0962
##    300       92.7738             nan     0.3896   -0.2250
##    320       90.7452             nan     0.3896   -0.1845
##    340       88.5841             nan     0.3896    0.0973
##    360       86.8098             nan     0.3896   -0.1892
##    380       85.0092             nan     0.3896   -0.1039
##    400       82.8555             nan     0.3896    0.0283
##    420       80.9717             nan     0.3896   -0.1535
##    440       79.2532             nan     0.3896   -0.0257
##    460       78.0586             nan     0.3896   -0.1740
##    480       76.7350             nan     0.3896    0.0409
##    500       75.1397             nan     0.3896   -0.1287
##    520       73.7946             nan     0.3896   -0.1500
##    540       72.4618             nan     0.3896   -0.0033
##    560       71.4492             nan     0.3896   -0.1380
##    580       70.4239             nan     0.3896   -0.0673
##    600       69.3448             nan     0.3896   -0.1040
##    620       68.3892             nan     0.3896   -0.1477
##    640       67.5616             nan     0.3896   -0.1621
##    660       66.4883             nan     0.3896   -0.1535
##    680       65.6748             nan     0.3896   -0.1677
##    700       64.6699             nan     0.3896   -0.0671
##    720       64.0246             nan     0.3896   -0.1522
##    740       63.2391             nan     0.3896   -0.1262
##    760       62.5489             nan     0.3896   -0.0833
##    780       61.8850             nan     0.3896   -0.1243
##    800       61.0850             nan     0.3896   -0.0690
##    820       60.3331             nan     0.3896   -0.0793
##    840       59.7439             nan     0.3896   -0.1801
##    860       59.0730             nan     0.3896   -0.0725
##    880       58.4614             nan     0.3896   -0.1300
##    900       57.8612             nan     0.3896   -0.0860
##    920       57.3315             nan     0.3896   -0.0956
##    940       56.6954             nan     0.3896   -0.0425
##    960       56.3049             nan     0.3896   -0.0876
##    980       55.7520             nan     0.3896   -0.1025
##   1000       55.0975             nan     0.3896   -0.0285
##   1020       54.4050             nan     0.3896   -0.1001
##   1040       53.8823             nan     0.3896   -0.0737
##   1060       53.1902             nan     0.3896   -0.0110
##   1080       52.7352             nan     0.3896   -0.1193
##   1100       52.2175             nan     0.3896   -0.0304
##   1120       51.8813             nan     0.3896   -0.2479
##   1140       51.6212             nan     0.3896   -0.1871
##   1160       51.1417             nan     0.3896   -0.0622
##   1180       50.5718             nan     0.3896   -0.0758
##   1200       50.2071             nan     0.3896   -0.0858
##   1220       49.7801             nan     0.3896   -0.0833
##   1240       49.2871             nan     0.3896   -0.0514
##   1260       48.9010             nan     0.3896   -0.0268
##   1280       48.5616             nan     0.3896   -0.0993
##   1300       48.1681             nan     0.3896   -0.0627
##   1320       47.9208             nan     0.3896   -0.0930
##   1340       47.6392             nan     0.3896   -0.0530
##   1360       47.3173             nan     0.3896   -0.1755
##   1380       46.9239             nan     0.3896   -0.1102
##   1400       46.7107             nan     0.3896   -0.2095
##   1420       46.3897             nan     0.3896   -0.0147
##   1440       46.1238             nan     0.3896   -0.0796
##   1460       45.8396             nan     0.3896   -0.1549
##   1480       45.5258             nan     0.3896   -0.0945
##   1500       45.2813             nan     0.3896   -0.1546
##   1520       44.9238             nan     0.3896   -0.0774
##   1540       44.6608             nan     0.3896   -0.0927
##   1560       44.4247             nan     0.3896   -0.1341
##   1580       44.0255             nan     0.3896   -0.1895
##   1600       43.8396             nan     0.3896   -0.0411
##   1620       43.6158             nan     0.3896   -0.1008
##   1640       43.2151             nan     0.3896   -0.0889
##   1660       42.9304             nan     0.3896   -0.0672
##   1680       42.7607             nan     0.3896   -0.0851
##   1700       42.4814             nan     0.3896   -0.0638
##   1720       42.2849             nan     0.3896   -0.1624
##   1740       42.0428             nan     0.3896   -0.0703
##   1760       41.8265             nan     0.3896   -0.0763
##   1780       41.6855             nan     0.3896   -0.0805
##   1800       41.5593             nan     0.3896   -0.0951
##   1820       41.3660             nan     0.3896   -0.0593
##   1840       41.1854             nan     0.3896   -0.0461
##   1860       40.9923             nan     0.3896   -0.0796
##   1880       40.7317             nan     0.3896   -0.1099
##   1900       40.6201             nan     0.3896   -0.1279
##   1920       40.4110             nan     0.3896   -0.1118
##   1940       40.1738             nan     0.3896   -0.0382
##   1960       40.0084             nan     0.3896   -0.0840
##   1980       39.8863             nan     0.3896   -0.0691
##   2000       39.7297             nan     0.3896   -0.0531
##   2020       39.5852             nan     0.3896   -0.0746
##   2040       39.3711             nan     0.3896   -0.0334
##   2060       39.1950             nan     0.3896   -0.0687
##   2080       39.0210             nan     0.3896   -0.0093
##   2100       38.8704             nan     0.3896   -0.0531
##   2120       38.7267             nan     0.3896   -0.2325
##   2140       38.4904             nan     0.3896   -0.0961
##   2160       38.3543             nan     0.3896   -0.1334
##   2180       38.1819             nan     0.3896   -0.0501
##   2200       38.0542             nan     0.3896   -0.1355
##   2220       37.9045             nan     0.3896   -0.1364
##   2240       37.7468             nan     0.3896   -0.1044
##   2260       37.6469             nan     0.3896   -0.0837
##   2280       37.5027             nan     0.3896   -0.0477
##   2300       37.4376             nan     0.3896   -0.0450
##   2320       37.3645             nan     0.3896   -0.0802
##   2340       37.2505             nan     0.3896   -0.0754
##   2360       37.0862             nan     0.3896   -0.1140
##   2380       36.9572             nan     0.3896   -0.0871
##   2400       36.8396             nan     0.3896   -0.1054
##   2420       36.7218             nan     0.3896   -0.1447
##   2440       36.5908             nan     0.3896   -0.0552
##   2460       36.4604             nan     0.3896   -0.0442
##   2480       36.2730             nan     0.3896   -0.0181
##   2500       36.1565             nan     0.3896   -0.0565
##   2520       36.0297             nan     0.3896   -0.0726
##   2540       35.8727             nan     0.3896   -0.0707
##   2560       35.7187             nan     0.3896   -0.0682
##   2580       35.6300             nan     0.3896   -0.0754
##   2600       35.5242             nan     0.3896   -0.0609
##   2620       35.4421             nan     0.3896   -0.0327
##   2640       35.3000             nan     0.3896   -0.0717
##   2660       35.2187             nan     0.3896   -0.0719
##   2680       35.1711             nan     0.3896   -0.0943
##   2700       35.1312             nan     0.3896   -0.0623
##   2720       35.0422             nan     0.3896   -0.0661
##   2740       34.9719             nan     0.3896   -0.0945
##   2760       34.8904             nan     0.3896   -0.0697
##   2780       34.7694             nan     0.3896   -0.0628
##   2800       34.6740             nan     0.3896   -0.0287
##   2820       34.6352             nan     0.3896   -0.0574
##   2840       34.5310             nan     0.3896   -0.0655
##   2860       34.4333             nan     0.3896   -0.0444
##   2880       34.3222             nan     0.3896   -0.0866
##   2900       34.3011             nan     0.3896   -0.0609
##   2920       34.1628             nan     0.3896   -0.0650
##   2940       34.0965             nan     0.3896   -0.0597
##   2960       34.0566             nan     0.3896   -0.0172
##   2980       33.9456             nan     0.3896   -0.0455
##   3000       33.8259             nan     0.3896   -0.0609
##   3020       33.7196             nan     0.3896   -0.0741
##   3040       33.6119             nan     0.3896   -0.0579
##   3060       33.5874             nan     0.3896   -0.1106
##   3080       33.4885             nan     0.3896   -0.0794
##   3100       33.4744             nan     0.3896   -0.1849
##   3120       33.3023             nan     0.3896   -0.0478
##   3140       33.2205             nan     0.3896   -0.0928
##   3160       33.1342             nan     0.3896   -0.0890
##   3180       33.0328             nan     0.3896   -0.0536
##   3200       32.9577             nan     0.3896   -0.0533
##   3220       32.8530             nan     0.3896   -0.0681
##   3240       32.8654             nan     0.3896   -0.0471
##   3260       32.7768             nan     0.3896   -0.0495
##   3280       32.7077             nan     0.3896   -0.0832
##   3300       32.6538             nan     0.3896   -0.0594
##   3320       32.5893             nan     0.3896   -0.0661
##   3340       32.5311             nan     0.3896   -0.0406
##   3360       32.5570             nan     0.3896   -0.0441
##   3380       32.4354             nan     0.3896   -0.0945
##   3400       32.3727             nan     0.3896   -0.0512
##   3420       32.3238             nan     0.3896   -0.0316
##   3440       32.2384             nan     0.3896   -0.0697
##   3460       32.1871             nan     0.3896   -0.0429
##   3480       32.0972             nan     0.3896   -0.0794
##   3500       31.9854             nan     0.3896   -0.0442
##   3520       31.9280             nan     0.3896   -0.0448
##   3540       31.8480             nan     0.3896   -0.0741
##   3560       31.8263             nan     0.3896   -0.1445
##   3580       31.7264             nan     0.3896   -0.0686
##   3600       31.6222             nan     0.3896   -0.0887
##   3620       31.6219             nan     0.3896   -0.1684
##   3640       31.5192             nan     0.3896   -0.0788
##   3660       31.4653             nan     0.3896   -0.0824
##   3680       31.3744             nan     0.3896   -0.1077
##   3700       31.3040             nan     0.3896   -0.0443
##   3720       31.3153             nan     0.3896   -0.1447
##   3740       31.2481             nan     0.3896   -0.0601
##   3760       31.1790             nan     0.3896   -0.0607
##   3780       31.2181             nan     0.3896   -0.5130
##   3800       31.1134             nan     0.3896   -0.0753
##   3820       30.9845             nan     0.3896   -0.0740
##   3840       30.9393             nan     0.3896   -0.0708
##   3860       30.9222             nan     0.3896   -0.0265
##   3880       30.8419             nan     0.3896   -0.0291
##   3900       30.8354             nan     0.3896   -0.0721
##   3920       30.7764             nan     0.3896   -0.1259
##   3940       30.7419             nan     0.3896   -0.1150
##   3960       30.6856             nan     0.3896   -0.0721
##   3980       30.6447             nan     0.3896   -0.0417
##   4000       30.5744             nan     0.3896   -0.0671
##   4020       30.5236             nan     0.3896   -0.0719
##   4040       30.4486             nan     0.3896   -0.0713
##   4060       30.4135             nan     0.3896   -0.0753
##   4080       30.3779             nan     0.3896   -0.1855
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      538.9561             nan     0.5470  636.7305
##      2      372.8056             nan     0.5470  169.2664
##      3      313.4867             nan     0.5470   57.0167
##      4      284.6830             nan     0.5470   26.5043
##      5      270.2090             nan     0.5470   14.4109
##      6      258.7196             nan     0.5470   11.0110
##      7      249.9269             nan     0.5470    7.2169
##      8      239.4164             nan     0.5470    8.9650
##      9      231.3752             nan     0.5470    6.9971
##     10      225.2811             nan     0.5470    5.1051
##     20      188.4264             nan     0.5470    2.1321
##     40      160.3528             nan     0.5470    0.7179
##     60      144.8292             nan     0.5470    0.0354
##     80      136.0276             nan     0.5470   -0.3044
##    100      125.8924             nan     0.5470   -0.1985
##    120      117.9102             nan     0.5470    0.0537
##    140      112.2190             nan     0.5470   -0.4669
##    160      106.8878             nan     0.5470   -0.1133
##    180      103.2165             nan     0.5470   -0.1942
##    200      100.2301             nan     0.5470   -0.4626
##    220       96.6519             nan     0.5470    0.0180
##    240       93.7008             nan     0.5470   -0.2279
##    260       90.4649             nan     0.5470   -0.0165
##    280       87.6059             nan     0.5470   -0.1844
##    300       85.5410             nan     0.5470   -0.1157
##    320       83.1453             nan     0.5470   -0.1128
##    340       81.1772             nan     0.5470   -0.1831
##    360       79.4130             nan     0.5470   -0.2509
##    380       77.6551             nan     0.5470    0.0630
##    400       76.1642             nan     0.5470   -0.2696
##    420       74.3502             nan     0.5470   -0.2899
##    440       73.2944             nan     0.5470   -0.3182
##    460       71.8284             nan     0.5470   -0.0697
##    480       70.1515             nan     0.5470   -0.1457
##    500       69.1432             nan     0.5470   -0.1301
##    520       67.7831             nan     0.5470   -0.1493
##    540       66.7442             nan     0.5470   -0.2217
##    560       65.9687             nan     0.5470   -0.1190
##    580       65.2866             nan     0.5470   -0.8582
##    600       63.9886             nan     0.5470   -0.0942
##    620       63.1216             nan     0.5470   -0.2450
##    640       62.1326             nan     0.5470   -0.2753
##    660       61.2606             nan     0.5470   -0.1086
##    680       60.4858             nan     0.5470   -0.2737
##    700       59.6035             nan     0.5470   -0.1940
##    720       58.8562             nan     0.5470   -0.1260
##    740       58.0577             nan     0.5470   -0.3069
##    760       57.5289             nan     0.5470   -0.3477
##    780       56.6959             nan     0.5470   -0.0680
##    800       56.1329             nan     0.5470   -0.1153
##    820       55.4822             nan     0.5470   -0.1391
##    840       54.8053             nan     0.5470   -0.1702
##    860       53.9178             nan     0.5470   -0.0820
##    880       53.5737             nan     0.5470   -0.0318
##    900       53.0706             nan     0.5470   -0.2020
##    920       52.6331             nan     0.5470   -0.1151
##    940       52.2214             nan     0.5470   -0.1514
##    960       51.6577             nan     0.5470   -0.2383
##    980       50.9177             nan     0.5470   -0.2011
##   1000       50.3114             nan     0.5470   -0.1761
##   1020       50.0623             nan     0.5470   -0.0690
##   1040       49.4910             nan     0.5470   -0.0865
##   1060       49.0702             nan     0.5470   -0.1771
##   1080       48.6451             nan     0.5470   -0.1555
##   1100       48.2811             nan     0.5470   -0.0687
##   1120       47.9946             nan     0.5470   -0.2420
##   1140       47.5489             nan     0.5470   -0.0638
##   1160       47.1475             nan     0.5470   -0.1326
##   1180       46.8413             nan     0.5470   -0.1945
##   1200       46.3953             nan     0.5470   -0.1621
##   1220       46.0213             nan     0.5470   -0.1926
##   1240       45.6132             nan     0.5470   -0.2429
##   1260       45.3134             nan     0.5470   -0.0775
##   1280       45.0545             nan     0.5470   -0.0866
##   1300       44.7921             nan     0.5470   -0.1168
##   1320       44.3905             nan     0.5470   -0.1146
##   1340       43.9337             nan     0.5470   -0.1765
##   1360       43.7999             nan     0.5470   -0.1624
##   1380       43.3672             nan     0.5470   -0.1511
##   1400       43.1658             nan     0.5470   -0.0998
##   1420       43.0283             nan     0.5470   -0.1919
##   1440       42.8793             nan     0.5470   -0.1314
##   1460       42.6555             nan     0.5470   -0.1814
##   1480       42.4577             nan     0.5470   -0.0925
##   1500       42.1597             nan     0.5470   -0.0928
##   1520       42.0237             nan     0.5470   -0.1920
##   1540       41.7537             nan     0.5470   -0.0323
##   1560       41.5616             nan     0.5470   -0.1722
##   1580       41.3772             nan     0.5470   -0.0436
##   1600       41.1877             nan     0.5470   -0.0736
##   1620       40.9182             nan     0.5470   -0.0917
##   1640       40.8450             nan     0.5470   -0.0878
##   1660       40.4850             nan     0.5470   -0.1034
##   1680       40.3796             nan     0.5470   -0.1729
##   1700       40.0781             nan     0.5470   -0.0295
##   1720       39.8253             nan     0.5470   -0.0903
##   1740       39.6966             nan     0.5470   -0.1137
##   1760       39.4600             nan     0.5470   -0.1501
##   1780       39.4242             nan     0.5470   -0.0625
##   1800       39.2284             nan     0.5470   -0.0951
##   1820       39.0761             nan     0.5470   -0.2690
##   1840       38.9349             nan     0.5470   -0.0929
##   1860       38.8005             nan     0.5470   -0.1060
##   1880       38.6005             nan     0.5470   -0.0999
##   1900       38.4188             nan     0.5470   -0.1228
##   1920       38.3166             nan     0.5470   -0.2488
##   1940       38.2092             nan     0.5470   -0.3127
##   1960       37.9593             nan     0.5470   -0.0577
##   1980       37.8165             nan     0.5470   -0.0970
##   2000       37.7148             nan     0.5470   -0.2030
##   2020       37.5467             nan     0.5470   -0.1078
##   2040       37.5087             nan     0.5470   -0.1176
##   2060       37.3156             nan     0.5470   -0.1616
##   2080       37.2686             nan     0.5470   -0.1534
##   2100       37.0329             nan     0.5470   -0.2425
##   2120       36.8316             nan     0.5470   -0.0462
##   2140       36.8374             nan     0.5470   -0.0858
##   2160       36.7047             nan     0.5470   -0.1194
##   2180       36.6385             nan     0.5470   -0.2225
##   2200       36.5006             nan     0.5470   -0.1453
##   2220       36.3954             nan     0.5470   -0.1459
##   2240       36.4256             nan     0.5470   -0.2324
##   2260       36.3313             nan     0.5470   -0.1100
##   2280       36.0971             nan     0.5470   -0.1543
##   2300       36.0353             nan     0.5470   -0.0260
##   2320       35.9727             nan     0.5470   -0.0617
##   2340       35.9430             nan     0.5470   -0.1435
##   2360       35.8695             nan     0.5470   -0.1958
##   2380       35.6545             nan     0.5470   -0.1495
##   2400       35.5225             nan     0.5470   -0.0909
##   2420       35.4419             nan     0.5470   -0.0932
##   2440       35.3119             nan     0.5470   -0.1483
##   2460       35.1598             nan     0.5470   -0.0995
##   2480       35.0076             nan     0.5470   -0.0703
##   2500       34.9642             nan     0.5470   -0.1402
##   2520       34.9331             nan     0.5470   -0.0457
##   2540       34.7587             nan     0.5470   -0.0977
##   2560       34.6945             nan     0.5470   -0.0517
##   2580       34.5330             nan     0.5470   -0.1000
##   2600       34.5991             nan     0.5470   -0.1824
##   2620       34.4184             nan     0.5470   -0.1209
##   2640       34.3530             nan     0.5470   -0.1355
##   2660       34.1998             nan     0.5470   -0.0546
##   2680       34.0698             nan     0.5470   -0.0940
##   2700       33.9838             nan     0.5470   -0.0425
##   2720       33.8366             nan     0.5470   -0.1449
##   2740       33.8005             nan     0.5470   -0.1071
##   2760       33.7960             nan     0.5470   -0.1808
##   2780       33.7187             nan     0.5470   -0.0380
##   2800       33.6996             nan     0.5470   -0.2309
##   2820       33.5618             nan     0.5470   -0.1126
##   2840       33.6004             nan     0.5470   -0.0810
##   2860       33.5039             nan     0.5470   -0.0750
##   2880       33.4186             nan     0.5470   -0.1165
##   2900       33.4167             nan     0.5470   -0.1119
##   2920       33.3660             nan     0.5470   -0.0688
##   2940       33.1700             nan     0.5470   -0.1216
##   2960       33.0975             nan     0.5470   -0.0861
##   2980       33.0973             nan     0.5470   -0.2253
##   3000       33.1051             nan     0.5470   -0.1028
##   3020       33.0100             nan     0.5470   -0.0472
##   3040       32.9987             nan     0.5470   -0.1237
##   3060       32.8058             nan     0.5470   -0.0533
##   3080       32.7599             nan     0.5470   -0.0551
##   3100       32.6907             nan     0.5470   -0.0401
##   3120       32.6287             nan     0.5470   -0.1362
##   3140       32.5585             nan     0.5470   -0.1315
##   3160       32.5624             nan     0.5470   -0.1241
##   3180       32.4214             nan     0.5470   -0.1756
##   3200       32.3856             nan     0.5470   -0.1198
##   3220       32.4105             nan     0.5470   -0.0655
##   3240       32.4140             nan     0.5470   -0.1539
##   3260       32.3337             nan     0.5470   -0.1326
##   3280       32.3275             nan     0.5470   -0.0801
##   3300       32.2570             nan     0.5470   -0.0442
##   3320       32.2882             nan     0.5470   -0.0528
##   3340       32.2261             nan     0.5470   -0.1130
##   3360       32.0559             nan     0.5470   -0.0406
##   3380       32.0355             nan     0.5470   -0.2084
##   3400       31.9923             nan     0.5470   -0.1296
##   3420       31.9452             nan     0.5470   -0.2047
##   3440       31.8718             nan     0.5470   -0.1210
##   3460       31.8225             nan     0.5470   -0.0629
##   3480       31.8046             nan     0.5470   -0.1712
##   3489       31.7651             nan     0.5470   -0.0707
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      821.2735             nan     0.2306  357.4904
##      2      604.8452             nan     0.2306  215.4831
##      3      463.5180             nan     0.2306  139.3365
##      4      373.6059             nan     0.2306   87.7389
##      5      316.4971             nan     0.2306   54.6248
##      6      277.7644             nan     0.2306   38.9442
##      7      250.4409             nan     0.2306   25.8902
##      8      232.1320             nan     0.2306   17.4474
##      9      216.2898             nan     0.2306   14.0407
##     10      205.7312             nan     0.2306   10.1643
##     20      162.9928             nan     0.2306    1.5304
##     40      132.5189             nan     0.2306    0.9219
##     60      116.8966             nan     0.2306    0.2388
##     80      105.4048             nan     0.2306   -0.0405
##    100       96.6068             nan     0.2306    0.0226
##    120       89.6807             nan     0.2306    0.0752
##    140       83.0713             nan     0.2306   -0.2003
##    160       77.9238             nan     0.2306   -0.1739
##    180       74.4981             nan     0.2306   -0.1720
##    200       71.1431             nan     0.2306   -0.0590
##    220       67.7049             nan     0.2306   -0.1382
##    240       65.0286             nan     0.2306   -0.1712
##    260       62.1297             nan     0.2306   -0.1159
##    280       59.8787             nan     0.2306    0.0329
##    300       58.1236             nan     0.2306   -0.0487
##    320       56.1503             nan     0.2306   -0.0887
##    340       54.5271             nan     0.2306   -0.0868
##    360       52.9863             nan     0.2306   -0.1056
##    380       51.5269             nan     0.2306   -0.1374
##    400       50.2964             nan     0.2306   -0.1628
##    420       49.1816             nan     0.2306   -0.0944
##    440       47.9502             nan     0.2306   -0.1871
##    460       46.9319             nan     0.2306   -0.2246
##    480       45.8832             nan     0.2306   -0.1970
##    500       44.9356             nan     0.2306   -0.2268
##    520       44.0038             nan     0.2306   -0.1062
##    540       43.1642             nan     0.2306   -0.0839
##    560       42.3752             nan     0.2306   -0.1416
##    580       41.7386             nan     0.2306   -0.1176
##    600       41.0385             nan     0.2306   -0.1092
##    620       40.3456             nan     0.2306   -0.1081
##    640       39.8137             nan     0.2306   -0.2815
##    660       39.1114             nan     0.2306   -0.0619
##    680       38.5531             nan     0.2306   -0.0913
##    700       37.9400             nan     0.2306   -0.0480
##    720       37.4194             nan     0.2306   -0.1096
##    740       36.8694             nan     0.2306   -0.1480
##    760       36.4566             nan     0.2306   -0.0110
##    780       35.9858             nan     0.2306   -0.1092
##    800       35.5873             nan     0.2306   -0.1181
##    820       35.2374             nan     0.2306   -0.0806
##    840       34.9362             nan     0.2306   -0.1044
##    860       34.5834             nan     0.2306   -0.1046
##    880       34.2821             nan     0.2306   -0.0581
##    900       33.9424             nan     0.2306   -0.1737
##    920       33.6521             nan     0.2306   -0.1606
##    940       33.4270             nan     0.2306   -0.2466
##    960       33.1245             nan     0.2306   -0.1591
##    980       32.8198             nan     0.2306   -0.1089
##   1000       32.5575             nan     0.2306   -0.0237
##   1020       32.2492             nan     0.2306   -0.1652
##   1040       32.0562             nan     0.2306   -0.1987
##   1060       31.8081             nan     0.2306   -0.0816
##   1080       31.6079             nan     0.2306   -0.1078
##   1100       31.4072             nan     0.2306   -0.1677
##   1120       31.2550             nan     0.2306   -0.1665
##   1140       31.0187             nan     0.2306   -0.0917
##   1160       30.7679             nan     0.2306   -0.1561
##   1180       30.5728             nan     0.2306   -0.1489
##   1200       30.3817             nan     0.2306   -0.1030
##   1220       30.2398             nan     0.2306   -0.2279
##   1240       30.0670             nan     0.2306   -0.1959
##   1260       29.8925             nan     0.2306   -0.1366
##   1280       29.7273             nan     0.2306   -0.0874
##   1300       29.5735             nan     0.2306   -0.1220
##   1320       29.4295             nan     0.2306   -0.1060
##   1340       29.2611             nan     0.2306   -0.1572
##   1360       29.1115             nan     0.2306   -0.1770
##   1380       28.9512             nan     0.2306   -0.1768
##   1400       28.8648             nan     0.2306   -0.0915
##   1420       28.7821             nan     0.2306   -0.1221
##   1440       28.6402             nan     0.2306   -0.1663
##   1460       28.4590             nan     0.2306   -0.1337
##   1480       28.3540             nan     0.2306   -0.0838
##   1500       28.2011             nan     0.2306   -0.1187
##   1520       28.0930             nan     0.2306   -0.0584
##   1540       28.0092             nan     0.2306   -0.2302
##   1560       27.8745             nan     0.2306   -0.1271
##   1580       27.8177             nan     0.2306   -0.0767
##   1600       27.6988             nan     0.2306   -0.1260
##   1620       27.5336             nan     0.2306   -0.0976
##   1640       27.4670             nan     0.2306   -0.1018
##   1660       27.3787             nan     0.2306   -0.1415
##   1680       27.3251             nan     0.2306   -0.1669
##   1700       27.2459             nan     0.2306   -0.1546
##   1720       27.1807             nan     0.2306   -0.0621
##   1740       27.1511             nan     0.2306   -0.1601
##   1760       27.0252             nan     0.2306   -0.0846
##   1780       26.9515             nan     0.2306   -0.1547
##   1800       26.8762             nan     0.2306   -0.1122
##   1820       26.8005             nan     0.2306   -0.1037
##   1840       26.7152             nan     0.2306   -0.0993
##   1860       26.6105             nan     0.2306   -0.1535
##   1880       26.5182             nan     0.2306   -0.1024
##   1900       26.4134             nan     0.2306   -0.0874
##   1920       26.3600             nan     0.2306   -0.1036
##   1940       26.2909             nan     0.2306   -0.1237
##   1960       26.2837             nan     0.2306   -0.0800
##   1980       26.1490             nan     0.2306   -0.0668
##   2000       26.1309             nan     0.2306   -0.1746
##   2020       26.0259             nan     0.2306   -0.1254
##   2040       25.9625             nan     0.2306   -0.1877
##   2060       25.8981             nan     0.2306   -0.0924
##   2080       25.8165             nan     0.2306   -0.0597
##   2100       25.7430             nan     0.2306   -0.1768
##   2120       25.7011             nan     0.2306   -0.1797
##   2140       25.6292             nan     0.2306   -0.1913
##   2160       25.5840             nan     0.2306   -0.0918
##   2180       25.4846             nan     0.2306   -0.0780
##   2200       25.4759             nan     0.2306   -0.1170
##   2220       25.3391             nan     0.2306   -0.1687
##   2240       25.2853             nan     0.2306   -0.0630
##   2260       25.2516             nan     0.2306   -0.1415
##   2280       25.1985             nan     0.2306   -0.0971
##   2300       25.1492             nan     0.2306   -0.0996
##   2320       25.1080             nan     0.2306   -0.0884
##   2340       25.0750             nan     0.2306   -0.2109
##   2360       25.0159             nan     0.2306   -0.0539
##   2380       24.9945             nan     0.2306   -0.0888
##   2400       24.9641             nan     0.2306   -0.0985
##   2420       24.9471             nan     0.2306   -0.0840
##   2440       24.9258             nan     0.2306   -0.1380
##   2460       24.9521             nan     0.2306   -0.2201
##   2480       24.8717             nan     0.2306   -0.1195
##   2500       24.7740             nan     0.2306   -0.1952
##   2520       24.7201             nan     0.2306   -0.1001
##   2540       24.6765             nan     0.2306   -0.0915
##   2560       24.5981             nan     0.2306   -0.1372
##   2580       24.5648             nan     0.2306   -0.0678
##   2600       24.5252             nan     0.2306   -0.1156
##   2620       24.5237             nan     0.2306   -0.2226
##   2640       24.4851             nan     0.2306   -0.1616
##   2660       24.4549             nan     0.2306   -0.1026
##   2680       24.4833             nan     0.2306   -0.1414
##   2700       24.4485             nan     0.2306   -0.0882
##   2720       24.3531             nan     0.2306   -0.1148
##   2740       24.3235             nan     0.2306   -0.1350
##   2760       24.2847             nan     0.2306   -0.1223
##   2780       24.2358             nan     0.2306   -0.0864
##   2800       24.2400             nan     0.2306   -0.1381
##   2820       24.2132             nan     0.2306   -0.1617
##   2840       24.1837             nan     0.2306   -0.0807
##   2860       24.1258             nan     0.2306   -0.1203
##   2880       24.0677             nan     0.2306   -0.1647
##   2900       24.0136             nan     0.2306   -0.1749
##   2920       24.0322             nan     0.2306   -0.1031
##   2940       23.9735             nan     0.2306   -0.0967
##   2960       23.9715             nan     0.2306   -0.1353
##   2980       23.9069             nan     0.2306   -0.1320
##   3000       23.8990             nan     0.2306   -0.1663
##   3020       23.8338             nan     0.2306   -0.1344
##   3040       23.8472             nan     0.2306   -0.1727
##   3060       23.8241             nan     0.2306   -0.1527
##   3080       23.7739             nan     0.2306   -0.1694
##   3100       23.7631             nan     0.2306   -0.1575
##   3120       23.7235             nan     0.2306   -0.1372
##   3140       23.6874             nan     0.2306   -0.1348
##   3160       23.6848             nan     0.2306   -0.1581
##   3180       23.6543             nan     0.2306   -0.1948
##   3200       23.6799             nan     0.2306   -0.0626
##   3220       23.5944             nan     0.2306   -0.1155
##   3240       23.5806             nan     0.2306   -0.1164
##   3260       23.5217             nan     0.2306   -0.0652
##   3280       23.5373             nan     0.2306   -0.1292
##   3300       23.4803             nan     0.2306   -0.0893
##   3320       23.4941             nan     0.2306   -0.0674
##   3340       23.5074             nan     0.2306   -0.1442
##   3360       23.4615             nan     0.2306   -0.0658
##   3380       23.4297             nan     0.2306   -0.1027
##   3400       23.4043             nan     0.2306   -0.1562
##   3420       23.4015             nan     0.2306   -0.1173
##   3440       23.3682             nan     0.2306   -0.1849
##   3460       23.3731             nan     0.2306   -0.1073
##   3480       23.3273             nan     0.2306   -0.1279
##   3500       23.3003             nan     0.2306   -0.0772
##   3520       23.2587             nan     0.2306   -0.1240
##   3540       23.2272             nan     0.2306   -0.0742
##   3560       23.1933             nan     0.2306   -0.0610
##   3580       23.1425             nan     0.2306   -0.1024
##   3600       23.1123             nan     0.2306   -0.1479
##   3620       23.0701             nan     0.2306   -0.1412
##   3640       23.0412             nan     0.2306   -0.1830
##   3660       23.1056             nan     0.2306   -0.0612
##   3680       23.0661             nan     0.2306   -0.0723
##   3700       23.0953             nan     0.2306   -0.2248
##   3720       23.0461             nan     0.2306   -0.1544
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      678.7697             nan     0.3896  503.1787
##      2      473.7464             nan     0.3896  204.6289
##      3      374.5152             nan     0.3896   96.9691
##      4      325.4351             nan     0.3896   47.2676
##      5      296.8331             nan     0.3896   29.2309
##      6      282.8396             nan     0.3896   13.2254
##      7      273.5464             nan     0.3896    8.1405
##      8      265.6337             nan     0.3896    6.7206
##      9      258.9429             nan     0.3896    5.5590
##     10      249.2657             nan     0.3896    8.6800
##     20      210.5262             nan     0.3896    1.0040
##     40      174.4130             nan     0.3896    0.4974
##     60      157.8671             nan     0.3896    0.2055
##     80      145.2970             nan     0.3896   -0.2202
##    100      135.6379             nan     0.3896    0.3472
##    120      127.5592             nan     0.3896   -0.1042
##    140      121.9216             nan     0.3896   -0.1611
##    160      116.9891             nan     0.3896   -0.0657
##    180      112.7789             nan     0.3896   -0.1919
##    200      109.5113             nan     0.3896   -0.2648
##    220      106.2491             nan     0.3896   -0.1419
##    240      102.7174             nan     0.3896   -0.1753
##    260      100.3076             nan     0.3896   -0.2457
##    280       98.1915             nan     0.3896   -0.1277
##    300       95.9194             nan     0.3896   -0.1300
##    320       93.5540             nan     0.3896   -0.0917
##    340       91.5172             nan     0.3896   -0.1340
##    360       89.4792             nan     0.3896   -0.1869
##    380       87.6528             nan     0.3896   -0.1588
##    400       86.1073             nan     0.3896   -0.1208
##    420       84.6157             nan     0.3896   -0.3042
##    440       83.1500             nan     0.3896   -0.2447
##    460       81.4441             nan     0.3896   -0.1307
##    480       80.2811             nan     0.3896   -0.1833
##    500       78.7158             nan     0.3896   -0.1092
##    520       77.8450             nan     0.3896   -0.0438
##    540       76.5861             nan     0.3896   -0.1696
##    560       75.3056             nan     0.3896   -0.2612
##    580       73.9105             nan     0.3896   -0.1296
##    600       73.1520             nan     0.3896   -0.2570
##    620       72.2562             nan     0.3896   -0.1377
##    640       71.3256             nan     0.3896   -0.1716
##    660       70.3970             nan     0.3896   -0.1325
##    680       69.5589             nan     0.3896   -0.1638
##    700       68.6190             nan     0.3896   -0.0585
##    720       67.8047             nan     0.3896   -0.1175
##    740       66.9259             nan     0.3896   -0.2552
##    760       66.2430             nan     0.3896   -0.1314
##    780       65.4954             nan     0.3896   -0.0568
##    800       64.6903             nan     0.3896   -0.1308
##    820       63.9675             nan     0.3896   -0.1172
##    840       63.3513             nan     0.3896   -0.2136
##    860       62.8535             nan     0.3896   -0.1228
##    880       62.2590             nan     0.3896   -0.1148
##    900       61.5711             nan     0.3896   -0.0429
##    920       61.0919             nan     0.3896   -0.1043
##    940       60.6473             nan     0.3896   -0.0869
##    960       60.0386             nan     0.3896   -0.0484
##    980       59.4946             nan     0.3896   -0.1240
##   1000       58.8383             nan     0.3896   -0.1157
##   1020       58.3256             nan     0.3896   -0.0937
##   1040       57.7263             nan     0.3896   -0.1029
##   1060       57.3149             nan     0.3896   -0.0688
##   1080       57.0129             nan     0.3896   -0.1248
##   1100       56.7178             nan     0.3896   -0.0949
##   1120       56.1230             nan     0.3896   -0.1216
##   1140       55.6542             nan     0.3896   -0.0873
##   1160       55.2210             nan     0.3896   -0.1439
##   1180       54.7752             nan     0.3896   -0.1886
##   1200       54.2996             nan     0.3896   -0.1128
##   1220       53.8903             nan     0.3896   -0.1219
##   1240       53.5837             nan     0.3896   -0.1591
##   1260       53.2952             nan     0.3896   -0.1530
##   1280       53.0067             nan     0.3896   -0.0781
##   1300       52.6151             nan     0.3896   -0.1468
##   1320       52.2411             nan     0.3896   -0.1849
##   1340       51.7698             nan     0.3896   -0.2023
##   1360       51.4921             nan     0.3896   -0.0616
##   1380       51.1168             nan     0.3896   -0.1087
##   1400       50.8279             nan     0.3896   -0.1054
##   1420       50.5784             nan     0.3896   -0.1712
##   1440       50.3571             nan     0.3896   -0.2900
##   1460       50.0429             nan     0.3896   -0.0670
##   1480       49.7357             nan     0.3896   -0.1057
##   1500       49.4614             nan     0.3896   -0.0632
##   1520       49.0396             nan     0.3896   -0.0912
##   1540       48.8563             nan     0.3896   -0.0754
##   1560       48.6417             nan     0.3896   -0.1915
##   1580       48.4499             nan     0.3896   -0.0879
##   1600       48.1809             nan     0.3896   -0.0871
##   1620       47.8943             nan     0.3896   -0.0379
##   1640       47.6609             nan     0.3896   -0.0941
##   1660       47.5230             nan     0.3896   -0.0630
##   1680       47.1871             nan     0.3896   -0.1621
##   1700       46.9943             nan     0.3896   -0.1446
##   1720       46.7638             nan     0.3896   -0.0579
##   1740       46.5576             nan     0.3896   -0.2008
##   1760       46.4219             nan     0.3896   -0.0927
##   1780       46.2024             nan     0.3896   -0.0917
##   1800       45.9276             nan     0.3896   -0.0284
##   1820       45.6417             nan     0.3896   -0.1532
##   1840       45.4427             nan     0.3896   -0.0667
##   1860       45.2601             nan     0.3896   -0.0989
##   1880       45.0145             nan     0.3896   -0.1633
##   1900       44.9091             nan     0.3896   -0.1027
##   1920       44.5459             nan     0.3896   -0.1513
##   1940       44.3379             nan     0.3896   -0.0445
##   1960       44.1742             nan     0.3896   -0.1214
##   1980       43.9255             nan     0.3896   -0.0640
##   2000       43.7273             nan     0.3896   -0.1098
##   2020       43.5350             nan     0.3896   -0.1110
##   2040       43.3342             nan     0.3896   -0.1293
##   2060       43.0723             nan     0.3896   -0.1037
##   2080       42.8954             nan     0.3896   -0.1281
##   2100       42.7022             nan     0.3896   -0.0712
##   2120       42.4837             nan     0.3896   -0.1386
##   2140       42.3553             nan     0.3896   -0.1329
##   2160       42.1339             nan     0.3896   -0.1351
##   2180       42.0330             nan     0.3896   -0.1058
##   2200       41.8915             nan     0.3896   -0.1018
##   2220       41.7346             nan     0.3896   -0.1206
##   2240       41.5618             nan     0.3896   -0.1436
##   2260       41.3557             nan     0.3896   -0.1109
##   2280       41.1918             nan     0.3896   -0.0482
##   2300       41.0792             nan     0.3896   -0.1395
##   2320       40.8760             nan     0.3896   -0.0376
##   2340       40.7965             nan     0.3896   -0.0965
##   2360       40.5925             nan     0.3896   -0.0687
##   2380       40.4811             nan     0.3896   -0.1498
##   2400       40.3469             nan     0.3896   -0.1134
##   2420       40.2298             nan     0.3896   -0.1467
##   2440       40.0854             nan     0.3896   -0.5026
##   2460       39.9772             nan     0.3896   -0.1457
##   2480       39.7427             nan     0.3896   -0.1953
##   2500       39.6248             nan     0.3896   -0.1268
##   2520       39.5458             nan     0.3896   -0.1192
##   2540       39.4648             nan     0.3896   -0.0748
##   2560       39.4122             nan     0.3896   -0.1052
##   2580       39.2953             nan     0.3896   -0.0934
##   2600       39.1861             nan     0.3896   -0.2995
##   2620       39.1411             nan     0.3896   -0.1109
##   2640       38.9863             nan     0.3896   -0.0694
##   2660       38.8770             nan     0.3896   -0.0796
##   2680       38.7394             nan     0.3896   -0.1144
##   2700       38.6302             nan     0.3896   -0.0656
##   2720       38.5204             nan     0.3896   -0.1515
##   2740       38.4068             nan     0.3896   -0.0789
##   2760       38.3130             nan     0.3896   -0.1821
##   2780       38.2335             nan     0.3896   -0.0887
##   2800       38.1104             nan     0.3896   -0.0881
##   2820       38.0686             nan     0.3896   -0.2139
##   2840       37.9391             nan     0.3896   -0.0260
##   2860       37.8168             nan     0.3896   -0.0968
##   2880       37.7827             nan     0.3896   -0.1478
##   2900       37.6899             nan     0.3896   -0.0366
##   2920       37.6350             nan     0.3896   -0.1538
##   2940       37.5536             nan     0.3896   -0.1585
##   2960       37.4543             nan     0.3896   -0.0876
##   2980       37.3442             nan     0.3896   -0.0879
##   3000       37.2774             nan     0.3896   -0.0968
##   3020       37.1292             nan     0.3896   -0.0695
##   3040       37.0499             nan     0.3896   -0.2096
##   3060       36.9116             nan     0.3896   -0.0915
##   3080       36.7656             nan     0.3896   -0.1112
##   3100       36.7063             nan     0.3896   -0.0415
##   3120       36.6446             nan     0.3896   -0.0586
##   3140       36.6195             nan     0.3896   -0.0913
##   3160       36.5575             nan     0.3896   -0.0911
##   3180       36.4092             nan     0.3896   -0.1764
##   3200       36.3781             nan     0.3896   -0.1029
##   3220       36.2858             nan     0.3896   -0.1165
##   3240       36.3171             nan     0.3896   -0.2225
##   3260       36.1502             nan     0.3896   -0.0785
##   3280       36.0805             nan     0.3896   -0.0377
##   3300       36.0263             nan     0.3896   -0.1591
##   3320       36.0143             nan     0.3896   -0.1038
##   3340       35.9272             nan     0.3896   -0.1381
##   3360       35.8829             nan     0.3896   -0.1128
##   3380       35.7968             nan     0.3896   -0.1384
##   3400       35.7431             nan     0.3896   -0.0686
##   3420       35.5831             nan     0.3896   -0.0942
##   3440       35.5588             nan     0.3896   -0.1479
##   3460       35.4656             nan     0.3896   -0.1668
##   3480       35.4356             nan     0.3896   -0.0920
##   3500       35.4153             nan     0.3896   -0.1039
##   3520       35.3517             nan     0.3896   -0.1287
##   3540       35.2191             nan     0.3896   -0.0881
##   3560       35.1673             nan     0.3896   -0.0672
##   3580       35.1571             nan     0.3896   -0.0747
##   3600       35.0397             nan     0.3896   -0.1368
##   3620       35.0185             nan     0.3896   -0.0379
##   3640       34.9427             nan     0.3896   -0.0541
##   3660       34.8696             nan     0.3896   -0.1064
##   3680       34.8058             nan     0.3896   -0.1344
##   3700       34.7319             nan     0.3896   -0.1003
##   3720       34.6140             nan     0.3896   -0.1123
##   3740       34.6236             nan     0.3896   -0.1274
##   3760       34.5303             nan     0.3896   -0.1421
##   3780       34.4588             nan     0.3896   -0.1580
##   3800       34.4329             nan     0.3896   -0.0543
##   3820       34.3954             nan     0.3896   -0.0977
##   3840       34.3589             nan     0.3896   -0.0535
##   3860       34.1564             nan     0.3896   -0.0865
##   3880       34.1175             nan     0.3896   -0.0482
##   3900       34.0697             nan     0.3896   -0.0601
##   3920       33.9709             nan     0.3896   -0.1098
##   3940       33.8426             nan     0.3896   -0.1177
##   3960       33.8286             nan     0.3896   -0.0703
##   3980       33.7368             nan     0.3896   -0.0579
##   4000       33.6601             nan     0.3896   -0.0548
##   4020       33.6051             nan     0.3896   -0.0419
##   4040       33.5224             nan     0.3896   -0.0898
##   4060       33.4392             nan     0.3896   -0.0639
##   4080       33.4156             nan     0.3896   -0.0326
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      539.3717             nan     0.5470  631.2045
##      2      376.1753             nan     0.5470  160.7672
##      3      315.0293             nan     0.5470   63.0649
##      4      288.4488             nan     0.5470   25.0061
##      5      269.4752             nan     0.5470   17.9580
##      6      261.9994             nan     0.5470    5.5769
##      7      251.0887             nan     0.5470   10.0484
##      8      244.1084             nan     0.5470    5.6937
##      9      236.7480             nan     0.5470    6.5185
##     10      230.6240             nan     0.5470    5.0779
##     20      194.3004             nan     0.5470    1.9813
##     40      163.4400             nan     0.5470   -0.5351
##     60      145.6628             nan     0.5470    0.4153
##     80      134.6843             nan     0.5470    0.1463
##    100      125.7745             nan     0.5470   -0.4416
##    120      119.8378             nan     0.5470    0.2066
##    140      113.8232             nan     0.5470    0.0744
##    160      110.1950             nan     0.5470   -0.6106
##    180      105.6077             nan     0.5470   -0.3508
##    200      102.3102             nan     0.5470   -0.3339
##    220       98.9151             nan     0.5470   -0.1948
##    240       95.9934             nan     0.5470   -0.2652
##    260       93.4396             nan     0.5470   -0.3648
##    280       91.1500             nan     0.5470   -0.5648
##    300       88.3665             nan     0.5470   -0.0977
##    320       86.2695             nan     0.5470   -0.2783
##    340       84.1983             nan     0.5470   -0.1763
##    360       82.7655             nan     0.5470   -0.3032
##    380       81.5338             nan     0.5470   -0.2236
##    400       80.0668             nan     0.5470   -0.3131
##    420       78.7997             nan     0.5470   -0.2772
##    440       77.8022             nan     0.5470   -0.3220
##    460       76.8174             nan     0.5470   -0.2371
##    480       75.3441             nan     0.5470   -0.0937
##    500       73.7480             nan     0.5470   -0.2069
##    520       72.2020             nan     0.5470   -0.1600
##    540       71.1509             nan     0.5470   -0.2522
##    560       69.9675             nan     0.5470   -0.2386
##    580       68.9143             nan     0.5470   -0.1792
##    600       67.9822             nan     0.5470   -0.0568
##    620       67.0970             nan     0.5470   -0.1015
##    640       66.2934             nan     0.5470   -0.1129
##    660       65.5200             nan     0.5470   -0.1376
##    680       64.3896             nan     0.5470   -0.1570
##    700       63.7005             nan     0.5470   -0.1308
##    720       62.8888             nan     0.5470   -0.4394
##    740       62.1984             nan     0.5470   -0.1997
##    760       61.2829             nan     0.5470   -0.3439
##    780       60.8896             nan     0.5470   -0.1560
##    800       60.3855             nan     0.5470   -0.0890
##    820       59.8354             nan     0.5470   -0.0976
##    840       59.2943             nan     0.5470   -0.1963
##    860       58.7244             nan     0.5470   -0.2676
##    880       58.1846             nan     0.5470   -0.3888
##    900       57.5478             nan     0.5470   -0.1353
##    920       57.3608             nan     0.5470   -0.3188
##    940       56.7997             nan     0.5470   -0.2773
##    960       56.5155             nan     0.5470   -0.2262
##    980       55.7275             nan     0.5470   -0.1064
##   1000       55.1192             nan     0.5470   -0.1389
##   1020       54.6636             nan     0.5470   -0.1855
##   1040       54.4902             nan     0.5470   -0.3892
##   1060       53.8056             nan     0.5470   -0.2056
##   1080       53.3693             nan     0.5470   -0.2398
##   1100       53.1831             nan     0.5470   -0.3139
##   1120       52.6977             nan     0.5470   -0.0342
##   1140       52.3858             nan     0.5470   -0.1137
##   1160       52.0616             nan     0.5470   -0.2084
##   1180       51.6934             nan     0.5470   -0.1681
##   1200       51.2339             nan     0.5470   -0.1660
##   1220       50.7999             nan     0.5470   -0.2539
##   1240       50.4964             nan     0.5470   -0.1433
##   1260       50.2016             nan     0.5470   -0.2425
##   1280       49.8479             nan     0.5470   -0.0641
##   1300       49.5744             nan     0.5470   -0.1779
##   1320       49.1965             nan     0.5470   -0.1465
##   1340       49.0167             nan     0.5470   -0.0837
##   1360       48.6209             nan     0.5470   -0.1192
##   1380       48.3427             nan     0.5470   -0.1238
##   1400       48.0050             nan     0.5470   -0.1750
##   1420       47.8191             nan     0.5470   -0.0830
##   1440       47.5355             nan     0.5470   -0.1014
##   1460       47.2261             nan     0.5470   -0.1554
##   1480       46.9824             nan     0.5470   -0.1652
##   1500       46.6864             nan     0.5470   -0.1267
##   1520       46.5472             nan     0.5470   -0.1375
##   1540       46.3337             nan     0.5470   -0.0594
##   1560       46.0624             nan     0.5470   -0.2621
##   1580       45.8643             nan     0.5470   -0.1908
##   1600       45.5966             nan     0.5470   -0.1320
##   1620       45.2768             nan     0.5470   -0.1757
##   1640       45.0164             nan     0.5470   -0.1344
##   1660       44.8505             nan     0.5470   -0.2830
##   1680       44.7594             nan     0.5470   -0.1252
##   1700       44.4879             nan     0.5470   -0.1448
##   1720       44.3574             nan     0.5470   -0.2027
##   1740       44.1611             nan     0.5470   -0.4781
##   1760       43.9728             nan     0.5470   -0.1156
##   1780       43.7677             nan     0.5470   -0.1337
##   1800       43.6085             nan     0.5470   -0.0900
##   1820       43.5138             nan     0.5470   -0.2330
##   1840       43.2687             nan     0.5470   -0.1189
##   1860       43.0960             nan     0.5470   -0.1741
##   1880       42.8766             nan     0.5470   -0.0694
##   1900       42.8654             nan     0.5470   -0.3317
##   1920       42.6844             nan     0.5470   -0.1579
##   1940       42.6166             nan     0.5470   -0.2553
##   1960       42.4922             nan     0.5470   -0.1928
##   1980       42.2491             nan     0.5470   -0.1232
##   2000       42.2270             nan     0.5470   -0.3343
##   2020       42.1839             nan     0.5470   -0.2488
##   2040       41.9087             nan     0.5470   -0.0914
##   2060       41.8098             nan     0.5470   -0.1732
##   2080       41.5401             nan     0.5470   -0.1326
##   2100       41.4335             nan     0.5470   -0.2241
##   2120       41.3115             nan     0.5470   -0.1848
##   2140       41.0302             nan     0.5470   -0.2476
##   2160       40.8373             nan     0.5470   -0.1795
##   2180       40.8055             nan     0.5470   -0.0908
##   2200       40.5014             nan     0.5470   -0.0913
##   2220       40.3324             nan     0.5470   -0.0610
##   2240       40.3085             nan     0.5470   -0.2031
##   2260       40.1569             nan     0.5470   -0.1532
##   2280       40.1237             nan     0.5470   -0.0690
##   2300       39.8960             nan     0.5470   -0.2168
##   2320       39.7334             nan     0.5470   -0.2628
##   2340       39.5415             nan     0.5470   -0.1281
##   2360       39.4126             nan     0.5470   -0.0822
##   2380       39.3370             nan     0.5470   -0.0972
##   2400       39.2413             nan     0.5470   -0.1166
##   2420       39.1292             nan     0.5470   -0.1067
##   2440       38.8574             nan     0.5470   -0.1275
##   2460       38.9329             nan     0.5470   -0.1321
##   2480       38.7404             nan     0.5470   -0.2108
##   2500       38.6735             nan     0.5470   -0.1856
##   2520       38.5802             nan     0.5470   -0.0910
##   2540       38.4766             nan     0.5470   -0.1158
##   2560       38.3017             nan     0.5470   -0.0786
##   2580       38.1582             nan     0.5470   -0.1294
##   2600       38.0466             nan     0.5470   -0.1932
##   2620       37.9306             nan     0.5470   -0.1703
##   2640       37.8771             nan     0.5470   -0.1866
##   2660       37.7699             nan     0.5470   -0.0938
##   2680       37.6040             nan     0.5470   -0.0619
##   2700       37.4931             nan     0.5470   -0.0845
##   2720       37.4839             nan     0.5470   -0.1321
##   2740       37.4054             nan     0.5470   -0.2117
##   2760       37.3062             nan     0.5470   -0.1956
##   2780       37.2358             nan     0.5470   -0.0513
##   2800       37.1819             nan     0.5470   -0.1570
##   2820       37.0921             nan     0.5470   -0.1781
##   2840       37.0707             nan     0.5470   -0.0735
##   2860       36.9389             nan     0.5470   -0.1656
##   2880       36.9391             nan     0.5470   -0.3601
##   2900       36.7907             nan     0.5470   -0.0483
##   2920       36.6721             nan     0.5470   -0.2078
##   2940       36.6139             nan     0.5470   -0.1680
##   2960       36.5902             nan     0.5470   -0.1129
##   2980       36.4781             nan     0.5470   -0.0999
##   3000       36.4549             nan     0.5470   -0.2363
##   3020       36.3832             nan     0.5470   -0.1402
##   3040       36.2836             nan     0.5470   -0.1785
##   3060       36.3109             nan     0.5470   -0.1586
##   3080       36.2759             nan     0.5470   -0.1022
##   3100       36.1986             nan     0.5470   -0.1935
##   3120       36.0797             nan     0.5470   -0.0494
##   3140       35.9071             nan     0.5470   -0.0376
##   3160       35.8620             nan     0.5470   -0.2380
##   3180       35.8125             nan     0.5470   -0.1707
##   3200       35.7069             nan     0.5470   -0.1614
##   3220       35.6845             nan     0.5470   -0.0889
##   3240       35.5877             nan     0.5470   -0.0352
##   3260       35.5528             nan     0.5470   -0.1009
##   3280       35.4826             nan     0.5470   -0.1602
##   3300       35.4631             nan     0.5470   -0.1250
##   3320       35.4777             nan     0.5470   -0.1728
##   3340       35.3689             nan     0.5470   -0.1461
##   3360       35.2346             nan     0.5470   -0.1459
##   3380       35.2076             nan     0.5470   -0.2063
##   3400       35.2724             nan     0.5470   -0.4826
##   3420       35.0515             nan     0.5470   -0.1591
##   3440       35.0133             nan     0.5470   -0.1146
##   3460       34.9285             nan     0.5470   -0.0936
##   3480       34.8377             nan     0.5470   -0.1669
##   3489       34.7579             nan     0.5470   -0.1585
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      819.8765             nan     0.2306  361.8322
##      2      598.2295             nan     0.2306  224.4198
##      3      464.0540             nan     0.2306  134.2621
##      4      376.0027             nan     0.2306   86.9478
##      5      317.5825             nan     0.2306   58.9315
##      6      278.5187             nan     0.2306   38.3842
##      7      251.2128             nan     0.2306   25.6245
##      8      230.2331             nan     0.2306   20.0655
##      9      216.3586             nan     0.2306   13.3059
##     10      205.3586             nan     0.2306    9.8532
##     20      162.4408             nan     0.2306    1.4088
##     40      133.9545             nan     0.2306    0.3749
##     60      116.6463             nan     0.2306    0.4550
##     80      104.9094             nan     0.2306    0.1072
##    100       95.5531             nan     0.2306    0.2552
##    120       88.0855             nan     0.2306    0.0193
##    140       82.6312             nan     0.2306   -0.1468
##    160       77.7783             nan     0.2306   -0.0623
##    180       73.8172             nan     0.2306   -0.1295
##    200       69.9356             nan     0.2306   -0.1116
##    220       66.1016             nan     0.2306   -0.1304
##    240       63.2089             nan     0.2306   -0.0773
##    260       60.9367             nan     0.2306   -0.1313
##    280       58.5218             nan     0.2306   -0.0858
##    300       56.4094             nan     0.2306   -0.1242
##    320       54.9835             nan     0.2306   -0.2025
##    340       53.1561             nan     0.2306   -0.1028
##    360       51.3590             nan     0.2306   -0.1202
##    380       49.9335             nan     0.2306   -0.1379
##    400       48.4651             nan     0.2306   -0.2069
##    420       47.1839             nan     0.2306   -0.1518
##    440       46.0711             nan     0.2306   -0.0609
##    460       45.2544             nan     0.2306   -0.1892
##    480       44.0896             nan     0.2306   -0.1681
##    500       43.1091             nan     0.2306   -0.1762
##    520       42.1875             nan     0.2306   -0.1269
##    540       41.3086             nan     0.2306   -0.0840
##    560       40.5454             nan     0.2306   -0.1971
##    580       39.9184             nan     0.2306   -0.1562
##    600       39.2210             nan     0.2306   -0.0071
##    620       38.5929             nan     0.2306   -0.1132
##    640       38.0082             nan     0.2306   -0.1333
##    660       37.4158             nan     0.2306   -0.0580
##    680       36.7591             nan     0.2306   -0.1318
##    700       36.1985             nan     0.2306   -0.0532
##    720       35.7535             nan     0.2306   -0.1721
##    740       35.1359             nan     0.2306   -0.1954
##    760       34.7109             nan     0.2306   -0.0614
##    780       34.2381             nan     0.2306   -0.1800
##    800       33.8372             nan     0.2306   -0.1747
##    820       33.4408             nan     0.2306   -0.1312
##    840       33.0957             nan     0.2306   -0.0842
##    860       32.7284             nan     0.2306   -0.0691
##    880       32.3847             nan     0.2306   -0.1260
##    900       32.0484             nan     0.2306   -0.0486
##    920       31.7746             nan     0.2306   -0.1386
##    940       31.5020             nan     0.2306   -0.1088
##    960       31.1952             nan     0.2306   -0.0896
##    980       30.9626             nan     0.2306   -0.2405
##   1000       30.7206             nan     0.2306   -0.0822
##   1020       30.4890             nan     0.2306   -0.1836
##   1040       30.2428             nan     0.2306   -0.1010
##   1060       29.9962             nan     0.2306   -0.0803
##   1080       29.7939             nan     0.2306   -0.1187
##   1100       29.5431             nan     0.2306   -0.1178
##   1120       29.4116             nan     0.2306   -0.1431
##   1140       29.1493             nan     0.2306   -0.1371
##   1160       28.9660             nan     0.2306   -0.0753
##   1180       28.7811             nan     0.2306   -0.0864
##   1200       28.5807             nan     0.2306   -0.1110
##   1220       28.4457             nan     0.2306   -0.2131
##   1240       28.2837             nan     0.2306   -0.1091
##   1260       28.1180             nan     0.2306   -0.1378
##   1280       27.9570             nan     0.2306   -0.1215
##   1300       27.7725             nan     0.2306   -0.0755
##   1320       27.6364             nan     0.2306   -0.1102
##   1340       27.5067             nan     0.2306   -0.0994
##   1360       27.4065             nan     0.2306   -0.0923
##   1380       27.2481             nan     0.2306   -0.0753
##   1400       27.0937             nan     0.2306   -0.1692
##   1420       26.9570             nan     0.2306   -0.1009
##   1440       26.8464             nan     0.2306   -0.1533
##   1460       26.6846             nan     0.2306   -0.0621
##   1480       26.6199             nan     0.2306   -0.0860
##   1500       26.5390             nan     0.2306   -0.0984
##   1520       26.3866             nan     0.2306   -0.1547
##   1540       26.2930             nan     0.2306   -0.0850
##   1560       26.2448             nan     0.2306   -0.1591
##   1580       26.1397             nan     0.2306   -0.0867
##   1600       25.9919             nan     0.2306   -0.0980
##   1620       25.9026             nan     0.2306   -0.1127
##   1640       25.7959             nan     0.2306   -0.1129
##   1660       25.7062             nan     0.2306   -0.0645
##   1680       25.6171             nan     0.2306   -0.0868
##   1700       25.5246             nan     0.2306   -0.1181
##   1720       25.3573             nan     0.2306   -0.2116
##   1740       25.2694             nan     0.2306   -0.1368
##   1760       25.1733             nan     0.2306   -0.0600
##   1780       25.0890             nan     0.2306   -0.1335
##   1800       25.0151             nan     0.2306   -0.1168
##   1820       24.9183             nan     0.2306   -0.1475
##   1840       24.8529             nan     0.2306   -0.2620
##   1860       24.7174             nan     0.2306   -0.0808
##   1880       24.6393             nan     0.2306   -0.1215
##   1900       24.5762             nan     0.2306   -0.1234
##   1920       24.5490             nan     0.2306   -0.1038
##   1940       24.4565             nan     0.2306   -0.1589
##   1960       24.4158             nan     0.2306   -0.0796
##   1980       24.3471             nan     0.2306   -0.0769
##   2000       24.2973             nan     0.2306   -0.1623
##   2020       24.2531             nan     0.2306   -0.0844
##   2040       24.1486             nan     0.2306   -0.0738
##   2060       24.1190             nan     0.2306   -0.1507
##   2080       24.0557             nan     0.2306   -0.1297
##   2100       24.0151             nan     0.2306   -0.1298
##   2120       23.9636             nan     0.2306   -0.0899
##   2140       23.9176             nan     0.2306   -0.1026
##   2160       23.8149             nan     0.2306   -0.0858
##   2180       23.7758             nan     0.2306   -0.0979
##   2200       23.7217             nan     0.2306   -0.1371
##   2220       23.6310             nan     0.2306   -0.0274
##   2240       23.6149             nan     0.2306   -0.1038
##   2260       23.5792             nan     0.2306   -0.0818
##   2280       23.5272             nan     0.2306   -0.1539
##   2300       23.4733             nan     0.2306   -0.1062
##   2320       23.4645             nan     0.2306   -0.1298
##   2340       23.3995             nan     0.2306   -0.0810
##   2360       23.3915             nan     0.2306   -0.2365
##   2380       23.3108             nan     0.2306   -0.1462
##   2400       23.2791             nan     0.2306   -0.1145
##   2420       23.2119             nan     0.2306   -0.0510
##   2440       23.2116             nan     0.2306   -0.0658
##   2460       23.1930             nan     0.2306   -0.1567
##   2480       23.1572             nan     0.2306   -0.0572
##   2500       23.1357             nan     0.2306   -0.0711
##   2520       23.0606             nan     0.2306   -0.0952
##   2540       22.9933             nan     0.2306   -0.1006
##   2560       22.9424             nan     0.2306   -0.1878
##   2580       22.9302             nan     0.2306   -0.1106
##   2600       22.8581             nan     0.2306   -0.0543
##   2620       22.8276             nan     0.2306   -0.0745
##   2640       22.7904             nan     0.2306   -0.1234
##   2660       22.7435             nan     0.2306   -0.1580
##   2680       22.7036             nan     0.2306   -0.1114
##   2700       22.7336             nan     0.2306   -0.2043
##   2720       22.6553             nan     0.2306   -0.1064
##   2740       22.6317             nan     0.2306   -0.1147
##   2760       22.6120             nan     0.2306   -0.1472
##   2780       22.5526             nan     0.2306   -0.1292
##   2800       22.5401             nan     0.2306   -0.0677
##   2820       22.5412             nan     0.2306   -0.0861
##   2840       22.5066             nan     0.2306   -0.1429
##   2860       22.4471             nan     0.2306   -0.1039
##   2880       22.4174             nan     0.2306   -0.0883
##   2900       22.4014             nan     0.2306   -0.1113
##   2920       22.3817             nan     0.2306   -0.0981
##   2940       22.3421             nan     0.2306   -0.1610
##   2960       22.2988             nan     0.2306   -0.1206
##   2980       22.3233             nan     0.2306   -0.0659
##   3000       22.3354             nan     0.2306   -0.0735
##   3020       22.2963             nan     0.2306   -0.2677
##   3040       22.2277             nan     0.2306   -0.1469
##   3060       22.2160             nan     0.2306   -0.1462
##   3080       22.1785             nan     0.2306   -0.1610
##   3100       22.1599             nan     0.2306   -0.0888
##   3120       22.1013             nan     0.2306   -0.1222
##   3140       22.1065             nan     0.2306   -0.0972
##   3160       22.1006             nan     0.2306   -0.1395
##   3180       22.0977             nan     0.2306   -0.0531
##   3200       22.0941             nan     0.2306   -0.0683
##   3220       22.0509             nan     0.2306   -0.0946
##   3240       22.0357             nan     0.2306   -0.1127
##   3260       22.0067             nan     0.2306   -0.0866
##   3280       21.9931             nan     0.2306   -0.1395
##   3300       21.9409             nan     0.2306   -0.0449
##   3320       21.9419             nan     0.2306   -0.2252
##   3340       21.8617             nan     0.2306   -0.1438
##   3360       21.8288             nan     0.2306   -0.0732
##   3380       21.8243             nan     0.2306   -0.1021
##   3400       21.7973             nan     0.2306   -0.0925
##   3420       21.7410             nan     0.2306   -0.1755
##   3440       21.7310             nan     0.2306   -0.0538
##   3460       21.6749             nan     0.2306   -0.1795
##   3480       21.6333             nan     0.2306   -0.0701
##   3500       21.6090             nan     0.2306   -0.1037
##   3520       21.6528             nan     0.2306   -0.2131
##   3540       21.6204             nan     0.2306   -0.1364
##   3560       21.6011             nan     0.2306   -0.1179
##   3580       21.5847             nan     0.2306   -0.2377
##   3600       21.5815             nan     0.2306   -0.0931
##   3620       21.5341             nan     0.2306   -0.0630
##   3640       21.5170             nan     0.2306   -0.0803
##   3660       21.4780             nan     0.2306   -0.0954
##   3680       21.5298             nan     0.2306   -0.0978
##   3700       21.4791             nan     0.2306   -0.0835
##   3720       21.4705             nan     0.2306   -0.1347
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      673.9031             nan     0.3896  500.2957
##      2      455.4256             nan     0.3896  216.0964
##      3      358.2185             nan     0.3896   95.1424
##      4      309.8896             nan     0.3896   47.2781
##      5      283.2738             nan     0.3896   25.4522
##      6      266.1795             nan     0.3896   16.0087
##      7      250.3216             nan     0.3896   14.0969
##      8      242.5260             nan     0.3896    7.8769
##      9      236.6881             nan     0.3896    5.4792
##     10      230.4422             nan     0.3896    5.6839
##     20      194.2343             nan     0.3896    1.0120
##     40      164.7300             nan     0.3896    0.8619
##     60      148.9902             nan     0.3896    0.1639
##     80      137.4391             nan     0.3896   -0.0071
##    100      129.5421             nan     0.3896   -0.2870
##    120      121.3237             nan     0.3896   -0.1098
##    140      116.5812             nan     0.3896   -0.1783
##    160      111.5619             nan     0.3896   -0.3795
##    180      107.8271             nan     0.3896   -0.1306
##    200      104.3140             nan     0.3896   -0.3044
##    220      100.8734             nan     0.3896   -0.1528
##    240       97.6328             nan     0.3896   -0.1211
##    260       95.2757             nan     0.3896   -0.0616
##    280       92.5858             nan     0.3896   -0.1949
##    300       90.0852             nan     0.3896   -0.1168
##    320       88.2393             nan     0.3896   -0.2722
##    340       86.2874             nan     0.3896   -0.0742
##    360       84.5821             nan     0.3896   -0.1664
##    380       82.8974             nan     0.3896   -0.0724
##    400       81.1764             nan     0.3896   -0.0969
##    420       79.6765             nan     0.3896   -0.3535
##    440       78.2627             nan     0.3896   -0.1443
##    460       76.8900             nan     0.3896   -0.2163
##    480       75.7343             nan     0.3896   -0.1056
##    500       74.2103             nan     0.3896   -0.1123
##    520       73.1068             nan     0.3896   -0.1164
##    540       71.8049             nan     0.3896   -0.0963
##    560       70.8381             nan     0.3896   -0.1184
##    580       69.6085             nan     0.3896   -0.1134
##    600       68.6873             nan     0.3896    0.0102
##    620       67.6131             nan     0.3896   -0.1351
##    640       66.6227             nan     0.3896   -0.1512
##    660       65.7311             nan     0.3896   -0.2059
##    680       64.8981             nan     0.3896   -0.2007
##    700       64.1122             nan     0.3896   -0.1255
##    720       63.3810             nan     0.3896   -0.0623
##    740       62.8539             nan     0.3896   -0.1732
##    760       62.1208             nan     0.3896   -0.2411
##    780       61.4335             nan     0.3896   -0.1531
##    800       60.8025             nan     0.3896   -0.0080
##    820       60.2368             nan     0.3896   -0.0954
##    840       59.4483             nan     0.3896   -0.1027
##    860       58.6776             nan     0.3896   -0.1093
##    880       58.3058             nan     0.3896   -0.1662
##    900       57.5642             nan     0.3896   -0.0783
##    920       56.9282             nan     0.3896   -0.1450
##    940       56.5008             nan     0.3896   -0.0904
##    960       56.0618             nan     0.3896   -0.1093
##    980       55.6596             nan     0.3896   -0.1709
##   1000       55.1015             nan     0.3896   -0.0890
##   1020       54.5777             nan     0.3896   -0.1376
##   1040       54.1125             nan     0.3896   -0.1166
##   1060       53.6697             nan     0.3896   -0.2745
##   1080       53.1394             nan     0.3896   -0.1658
##   1100       52.8326             nan     0.3896   -0.2075
##   1120       52.2829             nan     0.3896   -0.1101
##   1140       51.9782             nan     0.3896   -0.0896
##   1160       51.4099             nan     0.3896   -0.0612
##   1180       50.9002             nan     0.3896   -0.1619
##   1200       50.6390             nan     0.3896   -0.0915
##   1220       50.2102             nan     0.3896   -0.1754
##   1240       49.9869             nan     0.3896   -0.1358
##   1260       49.6688             nan     0.3896   -0.1224
##   1280       49.2512             nan     0.3896   -0.1184
##   1300       49.0323             nan     0.3896   -0.0517
##   1320       48.6898             nan     0.3896   -0.0957
##   1340       48.3388             nan     0.3896   -0.1299
##   1360       48.0321             nan     0.3896   -0.1919
##   1380       47.7014             nan     0.3896   -0.1383
##   1400       47.4831             nan     0.3896   -0.1025
##   1420       47.2306             nan     0.3896   -0.1188
##   1440       47.1072             nan     0.3896   -0.0826
##   1460       46.8655             nan     0.3896   -0.1188
##   1480       46.6476             nan     0.3896   -0.1930
##   1500       46.2485             nan     0.3896   -0.0328
##   1520       45.9593             nan     0.3896   -0.0029
##   1540       45.6805             nan     0.3896   -0.0828
##   1560       45.4350             nan     0.3896   -0.1041
##   1580       45.2622             nan     0.3896   -0.3048
##   1600       45.0871             nan     0.3896   -0.1827
##   1620       44.8916             nan     0.3896   -0.0988
##   1640       44.5624             nan     0.3896   -0.0544
##   1660       44.2683             nan     0.3896   -0.0989
##   1680       43.9999             nan     0.3896   -0.1739
##   1700       43.7996             nan     0.3896   -0.1818
##   1720       43.6401             nan     0.3896   -0.0999
##   1740       43.3890             nan     0.3896   -0.0940
##   1760       43.2293             nan     0.3896   -0.1083
##   1780       43.0082             nan     0.3896   -0.1001
##   1800       42.7921             nan     0.3896   -0.1731
##   1820       42.6738             nan     0.3896   -0.1008
##   1840       42.4238             nan     0.3896   -0.1324
##   1860       42.2028             nan     0.3896   -0.1106
##   1880       42.0306             nan     0.3896   -0.1672
##   1900       41.7389             nan     0.3896   -0.1013
##   1920       41.6192             nan     0.3896   -0.1463
##   1940       41.4178             nan     0.3896   -0.0847
##   1960       41.2259             nan     0.3896   -0.0570
##   1980       41.0306             nan     0.3896   -0.1492
##   2000       40.8690             nan     0.3896   -0.1067
##   2020       40.7217             nan     0.3896   -0.0946
##   2040       40.6477             nan     0.3896   -0.2288
##   2060       40.4525             nan     0.3896   -0.1078
##   2080       40.3123             nan     0.3896   -0.1624
##   2100       40.2133             nan     0.3896   -0.1123
##   2120       40.0696             nan     0.3896   -0.1103
##   2140       39.9090             nan     0.3896   -0.1153
##   2160       39.8397             nan     0.3896   -0.1601
##   2180       39.6905             nan     0.3896   -0.0474
##   2200       39.5341             nan     0.3896   -0.0628
##   2220       39.3501             nan     0.3896   -0.0955
##   2240       39.1995             nan     0.3896   -0.0612
##   2260       39.0361             nan     0.3896   -0.0470
##   2280       38.9398             nan     0.3896   -0.1767
##   2300       38.7996             nan     0.3896   -0.1557
##   2320       38.6542             nan     0.3896   -0.0701
##   2340       38.5434             nan     0.3896   -0.0616
##   2360       38.4232             nan     0.3896   -0.1369
##   2380       38.3354             nan     0.3896   -0.1177
##   2400       38.2002             nan     0.3896   -0.0640
##   2420       38.0974             nan     0.3896   -0.1246
##   2440       37.9927             nan     0.3896   -0.0709
##   2460       37.8229             nan     0.3896   -0.0926
##   2480       37.5999             nan     0.3896   -0.0659
##   2500       37.5115             nan     0.3896   -0.1029
##   2520       37.4422             nan     0.3896   -0.0773
##   2540       37.2669             nan     0.3896   -0.0394
##   2560       37.2045             nan     0.3896   -0.0621
##   2580       37.1051             nan     0.3896   -0.1339
##   2600       37.0232             nan     0.3896   -0.0317
##   2620       36.9208             nan     0.3896   -0.0977
##   2640       36.8658             nan     0.3896   -0.1219
##   2660       36.7807             nan     0.3896   -0.1460
##   2680       36.6335             nan     0.3896   -0.0238
##   2700       36.4513             nan     0.3896   -0.0257
##   2720       36.3483             nan     0.3896   -0.0796
##   2740       36.2104             nan     0.3896   -0.0329
##   2760       36.1544             nan     0.3896   -0.0935
##   2780       36.0480             nan     0.3896   -0.1642
##   2800       35.9996             nan     0.3896   -0.0881
##   2820       35.8920             nan     0.3896   -0.1034
##   2840       35.8172             nan     0.3896   -0.0776
##   2860       35.7377             nan     0.3896   -0.0943
##   2880       35.5685             nan     0.3896   -0.0618
##   2900       35.4652             nan     0.3896   -0.0431
##   2920       35.3710             nan     0.3896   -0.0478
##   2940       35.2530             nan     0.3896   -0.0681
##   2960       35.1941             nan     0.3896   -0.0964
##   2980       35.0997             nan     0.3896   -0.0743
##   3000       35.0411             nan     0.3896   -0.0958
##   3020       34.9146             nan     0.3896   -0.1337
##   3040       34.9076             nan     0.3896   -0.0920
##   3060       34.8664             nan     0.3896   -0.1061
##   3080       34.7441             nan     0.3896   -0.0414
##   3100       34.6495             nan     0.3896   -0.0659
##   3120       34.6451             nan     0.3896   -0.2376
##   3140       34.5214             nan     0.3896   -0.1166
##   3160       34.5298             nan     0.3896   -0.2053
##   3180       34.4098             nan     0.3896   -0.0448
##   3200       34.3777             nan     0.3896   -0.1166
##   3220       34.2705             nan     0.3896   -0.0598
##   3240       34.1637             nan     0.3896   -0.0947
##   3260       34.0411             nan     0.3896   -0.0405
##   3280       33.9546             nan     0.3896   -0.0693
##   3300       33.8490             nan     0.3896   -0.0949
##   3320       33.7447             nan     0.3896   -0.0989
##   3340       33.6874             nan     0.3896   -0.0559
##   3360       33.5369             nan     0.3896   -0.0936
##   3380       33.5626             nan     0.3896   -0.0733
##   3400       33.4417             nan     0.3896   -0.0806
##   3420       33.3873             nan     0.3896   -0.0809
##   3440       33.3274             nan     0.3896   -0.0831
##   3460       33.2429             nan     0.3896   -0.1112
##   3480       33.0975             nan     0.3896   -0.0490
##   3500       32.9955             nan     0.3896   -0.0809
##   3520       32.9825             nan     0.3896   -0.0294
##   3540       32.9453             nan     0.3896   -0.1194
##   3560       32.8637             nan     0.3896   -0.0783
##   3580       32.8467             nan     0.3896   -0.1302
##   3600       32.7580             nan     0.3896   -0.0525
##   3620       32.6683             nan     0.3896   -0.0769
##   3640       32.6107             nan     0.3896   -0.0835
##   3660       32.5858             nan     0.3896   -0.1847
##   3680       32.4910             nan     0.3896   -0.0526
##   3700       32.4548             nan     0.3896   -0.1408
##   3720       32.3622             nan     0.3896   -0.0413
##   3740       32.2923             nan     0.3896   -0.0640
##   3760       32.2367             nan     0.3896   -0.0802
##   3780       32.2408             nan     0.3896   -0.1045
##   3800       32.2339             nan     0.3896   -0.0776
##   3820       32.1961             nan     0.3896   -0.0477
##   3840       32.2337             nan     0.3896   -0.2785
##   3860       32.1050             nan     0.3896   -0.0942
##   3880       32.0525             nan     0.3896   -0.1175
##   3900       31.9834             nan     0.3896   -0.1561
##   3920       31.9580             nan     0.3896   -0.1473
##   3940       31.8972             nan     0.3896   -0.0689
##   3960       31.8321             nan     0.3896   -0.0928
##   3980       31.7670             nan     0.3896   -0.0869
##   4000       31.6861             nan     0.3896   -0.0904
##   4020       31.7139             nan     0.3896   -0.1142
##   4040       31.6407             nan     0.3896   -0.0590
##   4060       31.6072             nan     0.3896   -0.1380
##   4080       31.5295             nan     0.3896   -0.1541
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      543.4883             nan     0.5470  640.5988
##      2      373.4748             nan     0.5470  172.0838
##      3      315.5182             nan     0.5470   57.1656
##      4      282.7272             nan     0.5470   32.5032
##      5      266.6091             nan     0.5470   14.7395
##      6      253.4118             nan     0.5470   12.0296
##      7      244.1036             nan     0.5470    8.0444
##      8      237.6290             nan     0.5470    5.7521
##      9      231.7601             nan     0.5470    4.3249
##     10      226.2360             nan     0.5470    5.4128
##     20      195.5105             nan     0.5470    0.8227
##     40      163.9015             nan     0.5470    1.2057
##     60      149.5919             nan     0.5470   -0.6698
##     80      137.1281             nan     0.5470   -0.3005
##    100      129.1597             nan     0.5470   -0.0139
##    120      121.3919             nan     0.5470   -0.4264
##    140      115.8118             nan     0.5470   -0.0071
##    160      110.1566             nan     0.5470    0.0976
##    180      104.9477             nan     0.5470    0.0538
##    200      101.2229             nan     0.5470   -0.3254
##    220       98.2817             nan     0.5470   -0.1683
##    240       94.6229             nan     0.5470   -0.2898
##    260       92.3571             nan     0.5470   -0.3402
##    280       89.8366             nan     0.5470   -0.2887
##    300       87.7458             nan     0.5470   -0.0441
##    320       85.9497             nan     0.5470    0.1917
##    340       83.8035             nan     0.5470   -0.2869
##    360       81.3411             nan     0.5470   -0.2527
##    380       79.6566             nan     0.5470   -0.3291
##    400       78.3513             nan     0.5470   -0.4047
##    420       76.9632             nan     0.5470   -0.1788
##    440       75.5978             nan     0.5470   -0.1070
##    460       74.5064             nan     0.5470   -0.1740
##    480       73.2707             nan     0.5470   -0.2416
##    500       71.9748             nan     0.5470   -0.2122
##    520       70.7853             nan     0.5470   -0.2448
##    540       69.6390             nan     0.5470   -0.0512
##    560       68.8527             nan     0.5470   -0.1742
##    580       67.5710             nan     0.5470   -0.3371
##    600       66.7825             nan     0.5470   -0.1340
##    620       65.7939             nan     0.5470   -0.0869
##    640       64.8150             nan     0.5470   -0.0667
##    660       63.6550             nan     0.5470   -0.1584
##    680       63.0013             nan     0.5470   -0.3063
##    700       62.1851             nan     0.5470   -0.1820
##    720       61.4775             nan     0.5470   -0.2282
##    740       60.8895             nan     0.5470   -0.2705
##    760       60.1433             nan     0.5470   -0.2651
##    780       59.5230             nan     0.5470   -0.2296
##    800       58.7180             nan     0.5470   -0.1087
##    820       58.2527             nan     0.5470   -0.3931
##    840       57.6474             nan     0.5470   -0.1033
##    860       56.8383             nan     0.5470   -0.0729
##    880       56.3397             nan     0.5470   -0.1092
##    900       55.7581             nan     0.5470    0.0013
##    920       55.0518             nan     0.5470   -0.1712
##    940       54.4409             nan     0.5470   -0.0672
##    960       53.8321             nan     0.5470   -0.2265
##    980       53.4614             nan     0.5470   -0.0993
##   1000       53.0070             nan     0.5470   -0.1635
##   1020       52.6584             nan     0.5470   -0.1540
##   1040       52.3592             nan     0.5470   -0.2909
##   1060       51.8507             nan     0.5470   -0.1433
##   1080       51.4615             nan     0.5470   -0.1640
##   1100       50.8890             nan     0.5470   -0.1814
##   1120       50.6278             nan     0.5470   -0.2431
##   1140       50.2023             nan     0.5470    0.0025
##   1160       49.6968             nan     0.5470   -0.0787
##   1180       49.3723             nan     0.5470   -0.4154
##   1200       49.0834             nan     0.5470   -0.1259
##   1220       48.7116             nan     0.5470   -0.1851
##   1240       48.2451             nan     0.5470   -0.1798
##   1260       47.9123             nan     0.5470   -0.0727
##   1280       47.6716             nan     0.5470   -0.2048
##   1300       47.2868             nan     0.5470   -0.1093
##   1320       46.9720             nan     0.5470   -0.2103
##   1340       46.6454             nan     0.5470   -0.0733
##   1360       46.4821             nan     0.5470   -0.1588
##   1380       46.1590             nan     0.5470    0.0193
##   1400       45.9975             nan     0.5470   -0.1564
##   1420       45.6874             nan     0.5470   -0.1777
##   1440       45.4389             nan     0.5470   -0.1775
##   1460       45.2446             nan     0.5470   -0.2059
##   1480       44.9184             nan     0.5470    0.0429
##   1500       44.7380             nan     0.5470   -0.1351
##   1520       44.5496             nan     0.5470   -0.1632
##   1540       44.3400             nan     0.5470   -0.1109
##   1560       44.1774             nan     0.5470   -0.2138
##   1580       43.9117             nan     0.5470   -0.0722
##   1600       43.7284             nan     0.5470   -0.1555
##   1620       43.5310             nan     0.5470   -0.1258
##   1640       43.2050             nan     0.5470   -0.1549
##   1660       43.0115             nan     0.5470   -0.1106
##   1680       42.6874             nan     0.5470   -0.1546
##   1700       42.5185             nan     0.5470   -0.1168
##   1720       42.4123             nan     0.5470   -0.1529
##   1740       42.1947             nan     0.5470   -0.1621
##   1760       41.8812             nan     0.5470   -0.2319
##   1780       41.7542             nan     0.5470   -0.1417
##   1800       41.7704             nan     0.5470   -0.2929
##   1820       41.5211             nan     0.5470   -0.1072
##   1840       41.4024             nan     0.5470   -0.0981
##   1860       41.2777             nan     0.5470   -0.1112
##   1880       41.0254             nan     0.5470   -0.1512
##   1900       40.9554             nan     0.5470   -0.1331
##   1920       40.8087             nan     0.5470   -0.0371
##   1940       40.5816             nan     0.5470   -0.1994
##   1960       40.5270             nan     0.5470   -0.2801
##   1980       40.2339             nan     0.5470   -0.2085
##   2000       40.2109             nan     0.5470   -0.2033
##   2020       40.0113             nan     0.5470   -0.1482
##   2040       39.9071             nan     0.5470   -0.1813
##   2060       39.7081             nan     0.5470   -0.2135
##   2080       39.5292             nan     0.5470   -0.0738
##   2100       39.4006             nan     0.5470   -0.1676
##   2120       39.3752             nan     0.5470   -0.1183
##   2140       39.2138             nan     0.5470   -0.1998
##   2160       39.0565             nan     0.5470   -0.2488
##   2180       38.9535             nan     0.5470   -0.1670
##   2200       38.7862             nan     0.5470   -0.1419
##   2220       38.7810             nan     0.5470   -0.0892
##   2240       38.6108             nan     0.5470   -0.1938
##   2260       38.4350             nan     0.5470   -0.1385
##   2280       38.3410             nan     0.5470   -0.2381
##   2300       38.1556             nan     0.5470   -0.2041
##   2320       38.1520             nan     0.5470   -0.5389
##   2340       37.9377             nan     0.5470   -0.0636
##   2360       37.9202             nan     0.5470   -0.3013
##   2380       37.7444             nan     0.5470   -0.1175
##   2400       37.6146             nan     0.5470   -0.1423
##   2420       37.5770             nan     0.5470   -0.3117
##   2440       37.4795             nan     0.5470   -0.2243
##   2460       37.2826             nan     0.5470   -0.1735
##   2480       37.3931             nan     0.5470   -0.1606
##   2500       37.2550             nan     0.5470   -0.1283
##   2520       37.1331             nan     0.5470   -0.0610
##   2540       37.0108             nan     0.5470   -0.0873
##   2560       36.9261             nan     0.5470   -0.0555
##   2580       36.8990             nan     0.5470   -0.1864
##   2600       36.7652             nan     0.5470   -0.1779
##   2620       36.5827             nan     0.5470   -0.1643
##   2640       36.3740             nan     0.5470   -0.1601
##   2660       36.1991             nan     0.5470   -0.0011
##   2680       36.2092             nan     0.5470   -0.1509
##   2700       36.1222             nan     0.5470   -0.2528
##   2720       35.9968             nan     0.5470   -0.1764
##   2740       35.9361             nan     0.5470   -0.3287
##   2760       35.8817             nan     0.5470   -0.1684
##   2780       35.7672             nan     0.5470   -0.2022
##   2800       35.5962             nan     0.5470   -0.0684
##   2820       35.5327             nan     0.5470   -0.1635
##   2840       35.3906             nan     0.5470   -0.1656
##   2860       35.2345             nan     0.5470   -0.0917
##   2880       35.1213             nan     0.5470   -0.2014
##   2900       35.0848             nan     0.5470   -0.1745
##   2920       35.0482             nan     0.5470   -0.1855
##   2940       34.8702             nan     0.5470   -0.0538
##   2960       34.9113             nan     0.5470   -0.1491
##   2980       34.9093             nan     0.5470   -0.1049
##   3000       34.7691             nan     0.5470   -0.1936
##   3020       34.6902             nan     0.5470   -0.2690
##   3040       34.6935             nan     0.5470   -0.2933
##   3060       34.5735             nan     0.5470   -0.0343
##   3080       34.5341             nan     0.5470   -0.0850
##   3100       34.4176             nan     0.5470   -0.1013
##   3120       34.3971             nan     0.5470   -0.1389
##   3140       34.3392             nan     0.5470   -0.1242
##   3160       34.2075             nan     0.5470   -0.1675
##   3180       34.0633             nan     0.5470   -0.1467
##   3200       34.0964             nan     0.5470   -0.0459
##   3220       33.9575             nan     0.5470   -0.1594
##   3240       33.8873             nan     0.5470   -0.0880
##   3260       33.8063             nan     0.5470   -0.1030
##   3280       33.7274             nan     0.5470   -0.1162
##   3300       33.6564             nan     0.5470   -0.1451
##   3320       33.6013             nan     0.5470   -0.1953
##   3340       33.5415             nan     0.5470   -0.2858
##   3360       33.5413             nan     0.5470   -0.2274
##   3380       33.3867             nan     0.5470   -0.1829
##   3400       33.2448             nan     0.5470   -0.1552
##   3420       33.1647             nan     0.5470   -0.0840
##   3440       33.1560             nan     0.5470   -0.0778
##   3460       33.0858             nan     0.5470   -0.1247
##   3480       33.1523             nan     0.5470   -0.3534
##   3489       33.1368             nan     0.5470   -0.0527
## 
## Iter   TrainDeviance   ValidDeviance   StepSize   Improve
##      1      825.8791             nan     0.2306  352.7870
##      2      604.6337             nan     0.2306  218.9449
##      3      464.4869             nan     0.2306  136.9837
##      4      374.0772             nan     0.2306   90.3507
##      5      313.5929             nan     0.2306   59.2327
##      6      275.5003             nan     0.2306   36.9337
##      7      250.4041             nan     0.2306   24.3839
##      8      230.1951             nan     0.2306   18.6278
##      9      218.2181             nan     0.2306   10.8484
##     10      207.4607             nan     0.2306    9.6362
##     20      166.0889             nan     0.2306    1.0924
##     40      135.8218             nan     0.2306    0.7384
##     60      118.3438             nan     0.2306    0.3476
##     80      107.1410             nan     0.2306    0.2327
##    100       98.4464             nan     0.2306    0.0169
##    120       92.5357             nan     0.2306   -0.2377
##    140       86.5172             nan     0.2306   -0.0122
##    160       81.0686             nan     0.2306   -0.0907
##    180       76.6289             nan     0.2306    0.0360
##    200       72.6668             nan     0.2306   -0.3552
##    220       69.0827             nan     0.2306   -0.1137
##    240       66.4107             nan     0.2306   -0.2068
##    260       64.3094             nan     0.2306   -0.1655
##    280       62.0427             nan     0.2306   -0.0949
##    300       59.8466             nan     0.2306   -0.1726
##    320       58.0098             nan     0.2306   -0.0726
##    340       56.3326             nan     0.2306   -0.1098
##    360       54.7531             nan     0.2306   -0.0700
##    380       53.4447             nan     0.2306   -0.1428
##    400       52.1125             nan     0.2306   -0.0991
##    420       50.7526             nan     0.2306   -0.1202
##    440       49.4758             nan     0.2306   -0.1358
##    460       48.1515             nan     0.2306   -0.1350
##    480       47.0966             nan     0.2306   -0.1003
##    500       46.2223             nan     0.2306   -0.2348
##    520       45.2569             nan     0.2306   -0.1366
##    540       44.5323             nan     0.2306   -0.1584
##    560       43.7243             nan     0.2306   -0.1089
##    580       42.9962             nan     0.2306   -0.0628
##    600       42.3117             nan     0.2306   -0.2880
##    620       41.6916             nan     0.2306   -0.0902
##    640       41.0948             nan     0.2306   -0.1138
##    660       40.4783             nan     0.2306   -0.2369
##    680       39.8809             nan     0.2306   -0.0643
##    700       39.4089             nan     0.2306   -0.0395
##    720       38.8299             nan     0.2306   -0.1027
##    740       38.4079             nan     0.2306   -0.2828
##    760       37.8741             nan     0.2306   -0.1307
##    780       37.4330             nan     0.2306   -0.1424
##    800       37.0423             nan     0.2306   -0.1443
##    820       36.5754             nan     0.2306   -0.1626
##    840       36.0294             nan     0.2306   -0.0773
##    860       35.7887             nan     0.2306   -0.1432
##    880       35.4146             nan     0.2306   -0.1236
##    900       35.0992             nan     0.2306   -0.0323
##    920       34.7342             nan     0.2306   -0.0464
##    940       34.4255             nan     0.2306   -0.1259
##    960       34.0454             nan     0.2306   -0.1622
##    980       33.8401             nan     0.2306   -0.0893
##   1000       33.5599             nan     0.2306   -0.0910
##   1020       33.3325             nan     0.2306   -0.0565
##   1040       33.1393             nan     0.2306   -0.1478
##   1060       32.8681             nan     0.2306   -0.1520
##   1080       32.5842             nan     0.2306   -0.1416
##   1100       32.3639             nan     0.2306   -0.0870
##   1120       32.1349             nan     0.2306   -0.1212
##   1140       31.8946             nan     0.2306   -0.1494
##   1160       31.6717             nan     0.2306   -0.1276
##   1180       31.5442             nan     0.2306   -0.1497
##   1200       31.3740             nan     0.2306   -0.1287
##   1220       31.1789             nan     0.2306   -0.0825
##   1240       30.9754             nan     0.2306   -0.1239
##   1260       30.8236             nan     0.2306   -0.0916
##   1280       30.6417             nan     0.2306   -0.1834
##   1300       30.4654             nan     0.2306   -0.1392
##   1320       30.2752             nan     0.2306   -0.0938
##   1340       30.1238             nan     0.2306   -0.1065
##   1360       29.9890             nan     0.2306   -0.0869
##   1380       29.8416             nan     0.2306   -0.0681
##   1400       29.7993             nan     0.2306   -0.1484
##   1420       29.6922             nan     0.2306   -0.0935
##   1440       29.5852             nan     0.2306   -0.1404
##   1460       29.4424             nan     0.2306   -0.1369
##   1480       29.3890             nan     0.2306   -0.1399
##   1500       29.2086             nan     0.2306   -0.0919
##   1520       29.1050             nan     0.2306   -0.1738
##   1540       29.0109             nan     0.2306   -0.0985
##   1560       28.9058             nan     0.2306   -0.1659
##   1580       28.8039             nan     0.2306   -0.1418
##   1600       28.6836             nan     0.2306   -0.1439
##   1620       28.6034             nan     0.2306   -0.1470
##   1640       28.4980             nan     0.2306   -0.1120
##   1660       28.3910             nan     0.2306   -0.1058
##   1680       28.2499             nan     0.2306   -0.1360
##   1700       28.0951             nan     0.2306   -0.1140
##   1720       28.0154             nan     0.2306   -0.0764
##   1740       27.9393             nan     0.2306   -0.1213
##   1760       27.8366             nan     0.2306   -0.1516
##   1780       27.8112             nan     0.2306   -0.0817
##   1800       27.7443             nan     0.2306   -0.1050
##   1820       27.6864             nan     0.2306   -0.1282
##   1840       27.5787             nan     0.2306   -0.0507
##   1860       27.5582             nan     0.2306   -0.0937
##   1880       27.4412             nan     0.2306   -0.1076
##   1900       27.3738             nan     0.2306   -0.0454
##   1920       27.2789             nan     0.2306   -0.0956
##   1940       27.1815             nan     0.2306   -0.1392
##   1960       27.1028             nan     0.2306   -0.0820
##   1980       27.0637             nan     0.2306   -0.1511
##   2000       27.0235             nan     0.2306   -0.1028
##   2020       26.9347             nan     0.2306   -0.1278
##   2040       26.8404             nan     0.2306   -0.0872
##   2060       26.7778             nan     0.2306   -0.1074
##   2080       26.7048             nan     0.2306   -0.1787
##   2100       26.6610             nan     0.2306   -0.1760
##   2120       26.5537             nan     0.2306   -0.1463
##   2140       26.5259             nan     0.2306   -0.2061
##   2160       26.4604             nan     0.2306   -0.1322
##   2180       26.4076             nan     0.2306   -0.1003
##   2200       26.3905             nan     0.2306   -0.1337
##   2220       26.2713             nan     0.2306   -0.1163
##   2240       26.2807             nan     0.2306   -0.1762
##   2260       26.1945             nan     0.2306   -0.1502
##   2280       26.0970             nan     0.2306   -0.0850
##   2300       26.0312             nan     0.2306   -0.0921
##   2320       26.0160             nan     0.2306   -0.1000
##   2340       26.0149             nan     0.2306   -0.1588
##   2360       25.9428             nan     0.2306   -0.1499
##   2380       25.8676             nan     0.2306   -0.0538
##   2400       25.8622             nan     0.2306   -0.1628
##   2420       25.8055             nan     0.2306   -0.2000
##   2440       25.7835             nan     0.2306   -0.1242
##   2460       25.7219             nan     0.2306   -0.0902
##   2480       25.6740             nan     0.2306   -0.1047
##   2500       25.6816             nan     0.2306   -0.2449
##   2520       25.5883             nan     0.2306   -0.1242
##   2540       25.5543             nan     0.2306   -0.2254
##   2560       25.4758             nan     0.2306   -0.1299
##   2580       25.4492             nan     0.2306   -0.1239
##   2600       25.3781             nan     0.2306   -0.0886
##   2620       25.3923             nan     0.2306   -0.1587
##   2640       25.3381             nan     0.2306   -0.1267
##   2660       25.2311             nan     0.2306   -0.0770
##   2680       25.2554             nan     0.2306   -0.0864
##   2700       25.1934             nan     0.2306   -0.1398
##   2720       25.1665             nan     0.2306   -0.0966
##   2740       25.1568             nan     0.2306   -0.1808
##   2760       25.1023             nan     0.2306   -0.0710
##   2780       25.1075             nan     0.2306   -0.1114
##   2800       25.0579             nan     0.2306   -0.1997
##   2820       25.0161             nan     0.2306   -0.0947
##   2840       24.9547             nan     0.2306   -0.1378
##   2860       24.9290             nan     0.2306   -0.1060
##   2880       24.8519             nan     0.2306   -0.1242
##   2900       24.8170             nan     0.2306   -0.1081
##   2920       24.8006             nan     0.2306   -0.1495
##   2940       24.8130             nan     0.2306   -0.1046
##   2960       24.7694             nan     0.2306   -0.1820
##   2980       24.7572             nan     0.2306   -0.1126
##   3000       24.7522             nan     0.2306   -0.1239
##   3020       24.6830             nan     0.2306   -0.1086
##   3040       24.6292             nan     0.2306   -0.1145
##   3060       24.6154             nan     0.2306   -0.1055
##   3080       24.5643             nan     0.2306   -0.0991
##   3100       24.5155             nan     0.2306   -0.1310
##   3120       24.5281             nan     0.2306   -0.1426
##   3140       24.4493             nan     0.2306   -0.1086
##   3160       24.4859             nan     0.2306   -0.0969
##   3180       24.4969             nan     0.2306   -0.2362
##   3200       24.4491             nan     0.2306   -0.0907
##   3220       24.3993             nan     0.2306   -0.1408
##   3240       24.3398             nan     0.2306   -0.0870
##   3260       24.3484             nan     0.2306   -0.1428
##   3280       24.3045             nan     0.2306   -0.1289
##   3300       24.2932             nan     0.2306   -0.0766
##   3320       24.1917             nan     0.2306   -0.1435
##   3340       24.2219             nan     0.2306   -0.1258
##   3360       24.1798             nan     0.2306   -0.0573
##   3380       24.1487             nan     0.2306   -0.1043
##   3400       24.1712             nan     0.2306   -0.0492
##   3420       24.0916             nan     0.2306   -0.0456
##   3440       24.0528             nan     0.2306   -0.0817
##   3460       24.0614             nan     0.2306   -0.0918
##   3480       24.0629             nan     0.2306   -0.2024
##   3500       24.0164             nan     0.2306   -0.1179
##   3520       24.0062             nan     0.2306   -0.1288
##   3540       23.9972             nan     0.2306   -0.1249
##   3560       23.9713             nan     0.2306   -0.0980
##   3580       23.9605             nan     0.2306   -0.1061
##   3600       23.9468             nan     0.2306   -0.1034
##   3620       23.9403             nan     0.2306   -0.0822
##   3640       23.9423             nan     0.2306   -0.0816
##   3660       23.8858             nan     0.2306   -0.1256
##   3680       23.9108             nan     0.2306   -0.0521
##   3700       23.8608             nan     0.2306   -0.0925
##   3720       23.8501             nan     0.2306   -0.1449
gb_model$results
##   shrinkage interaction.depth n.minobsinnode n.trees     RMSE  Rsquared
## 3 0.5469976                 3              9    3489 11.15145 0.8960526
## 1 0.2306400                 8              8    3720 10.30007 0.9107268
## 2 0.3895644                 3             16    4080 10.50872 0.9071559
##        MAE    RMSESD  RsquaredSD     MAESD
## 3 6.882001 0.4931114 0.008629503 0.1895580
## 1 6.056816 0.4177467 0.007095070 0.1247801
## 2 6.438167 0.3027341 0.005237252 0.1097184
gb_model$bestTune
##   n.trees interaction.depth shrinkage n.minobsinnode
## 1    3720                 8   0.23064              8
gb_model$finalModel
## A gradient boosted model with gaussian loss function.
## 3720 iterations were performed.
## There were 81 predictors of which 81 had non-zero influence.

Generalization of Error

# Predicting Tc for training/test
pred_test_gb = predict(gb_model, newdata = test_set)
rmse_test_gb <- RMSE(pred_test_gb,test_set$critical_temp)

# Calculates RMSE of training pred
cat("\nGradient Boosting Model: RMSE for the test predictions =", rmse_test_gb)
## 
## Gradient Boosting Model: RMSE for the test predictions = 10.84253

Visualization of Gernerailzed Error

# Visualizing the fit
gb_test <- ggplot() +
  geom_point(aes(x = test_set$critical_temp, y = pred_test_gb),
            colour = 'deeppink1',alpha=0.5,size=3) +
  ggtitle('Gradient Boosting') +
  ylab('Prediction') +
  xlab('True Value (Tc)') + 
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

gb_test

3.4 K-Nearest-Neighbor Regression

KNN is a non-parametric and instance-based learning, and the only tuning parameter in this algorithm is k, the number neighbors (closest training examples) in the feature space. To make a prediction for an unseen datapoint (a query vector), we simply find the k nearest training examples with the smnallest Euclidean distance to it and take the average as output.

knn_model <- train(critical_temp ~ ., 
                   method = "knn",
                   tuneGrid = expand.grid(k = c(1:5)), trControl = trainControl(method = "cv", number = 10),
                   data = training_set,
                   preProc = c("center", "scale","BoxCox"))

knn_model$results
##   k     RMSE  Rsquared      MAE    RMSESD  RsquaredSD     MAESD
## 1 1 10.82027 0.9028101 5.531605 0.5105083 0.007880347 0.2159699
## 2 2 10.39815 0.9092257 5.457188 0.4352636 0.006771136 0.1920420
## 3 3 10.52456 0.9067717 5.684855 0.3819896 0.006198385 0.2393790
## 4 4 10.77315 0.9021655 5.934506 0.3410619 0.005543605 0.2100993
## 5 5 10.94480 0.8989689 6.091431 0.3629183 0.005911341 0.2088961
knn_model$bestTune
##   k
## 2 2
knn_model$finalModel
## 2-nearest neighbor regression model

Generalization of Error

# Predicting Tc for training/test
pred_test_knn = predict(knn_model, newdata = test_set)
rmse_test_knn <- RMSE(pred_test_knn,test_set$critical_temp)
# Calculates RMSE of training pred
cat("\nKNN Model: RMSE for the test predictions =", rmse_test_knn)
## 
## KNN Model: RMSE for the test predictions = 11.80047

Visualization of Gernerailzed Error

# Visualizing the fit
knn_test <- ggplot() +
  geom_point(aes(x = test_set$critical_temp, y = pred_test_knn),
            colour = 'red',alpha=0.5,size=3) +
  ggtitle('K Nearest Neighbor') +
  ylab('Prediction') +
  xlab('True Value (Tc)')  +
  theme_minimal() + 
  geom_abline(colour = "grey80", size = 1)

knn_test

4. Model Comparison

model <- c("Liner_regression",
            "Lasso_regression",
            "Linear_reg_featureCrosses",
            "Random_forest",
            "Gradient_boosting",
            "KNN"
            )

rmse <- c(round(rmse_test1.3,4),
                      round(rmse_test2.2,4),
                      "HUGE (Overfitting)",
                      round(rmse_test_rf,4),
                      round(rmse_test_gb,4),
                      round(rmse_test_knn,4))


models.test.metrics <- data.frame(model,rmse) 
models.test.metrics
##                       model               rmse
## 1          Liner_regression            18.4647
## 2          Lasso_regression            16.5828
## 3 Linear_reg_featureCrosses HUGE (Overfitting)
## 4             Random_forest            10.2137
## 5         Gradient_boosting            10.8425
## 6                       KNN            11.8005

Putting them all together, we can tell that the non-linear models clearly did a better job in general. At least the relationship between the true values and the predictions is much stronger. Unlike our linear regression models, there isn’t a gap or a shift in between the Y=X trend.

plot_results <- grid.arrange(Linear_regression_test + theme(plot.title = element_text(size = 10),
                                            axis.title.x = element_text(size=7),
                                            axis.title.y = element_text(size=7)),
             
             Lasso_regression_test + theme(plot.title = element_text(size = 10),
                                           axis.title.x = element_text(size=7),
                                           axis.title.y = element_text(size=7)),
             
             featureCrosses_test + theme(plot.title = element_text(size = 10),
                                         axis.title.x = element_text(size=7),
                                         axis.title.y = element_text(size=7)),
             
             rf_test + theme(plot.title = element_text(size = 10),
                             axis.title.x = element_text(size=7),
                             axis.title.y = element_text(size=7)),
             
             gb_test + theme(plot.title = element_text(size = 10),
                             axis.title.x = element_text(size=7),
                             axis.title.y = element_text(size=7)),
             
             knn_test + theme(plot.title = element_text(size = 10),
                              axis.title.x = element_text(size=7),
                             axis.title.y = element_text(size=7)),
             nrow=2)

5. Conclusion

In the beginning we explore the relationships between features and the label. We observed correlations between some of the features, however, there seemed to be no obvious linear relationship between features and the label. So then it makes sense that the linear regression models perform poorly while other non-linear models such as Random Forest and Gradient Boosting outperformed its counterpart using clever methods like bagging and boosting. Even thought we tried encoding the nonlinearity and got a better predictive power for the linear regression, we still suffered from overfitting. Finally, we saw that KNN, the non-parametric model, performs pretty well on the test data, just a little worse than those ensenble learning models. To sum up, Random_forest was the best model while the overfitting linear regression (with thousands of feature crosses) was the worst.

5. Discussion

We have gone through different scenarios in this project. We first tried to improve the linear regression models through feature engineering, feature transformation, and then we dealth with an overfitting model that had thousands of engineered features. Finally, we realised non-linear models, even with raw input data, significantly outperformed any kind of linear regression models. It tells us that a non-linear problem requires a non-linear solution.

Also, all three of the non-linear models performed well with just the full set of raw features, we can still expect them to perform maybe slightly better with the pre-processed features. For example, for algorithms that rely on calculating distance for classification or regression such as KNN, if the feature space is too sparse or too large, calculating the Euclidean distance and searching for neighbors becomes inefficient and this algorithm may suffer from the curse of dimensionality where all vectors are almost equidistant to the search query vector. In this case, pre-processing steps such as feature extraction and dimension reduction performed on the full size raw data can help avoid the effects of the curse of dimensionality.

Reference

  1. https://machinelearningmastery.com/feature-selection-with-the-caret-r-package/
  2. https://data.library.virginia.edu/diagnostic-plots/#targetText=Scale%2DLocation,equally%20(randomly)%20spread%20points.
  3. https://www.coursera.org/learn/ml-regression/home/welcome
  4. https://www.youtube.com/watch?v=ctmNq7FgbvI
  5. https://en.wikipedia.org/wiki/Gradient_boosting
  6. https://www.nature.com/subjects/superconductors